LLM Output Validation: Schema-Constrained Generation in Production

Reliable structured LLM output takes four layers: constrain generation with the provider's structured outputs or strict tool use, validate meaning in your own code (Pydantic, Zod), retry once or twice with the exact validation error, and define a fallback for when repair fails. Prompting "please return JSON" is not a layer; it is a hope.

The demo always works: you ask for JSON, get JSON, and JSON.parse it. Then real traffic arrives and a small fraction of responses come back with a trailing comment, a markdown code fence, a hallucinated extra field, or a number where you expected a string. At a million requests a day, even a fraction of a percent is thousands of broken responses. The layers below are how you stop getting paged for it.

Stop asking nicely; constrain the output

The single biggest win is moving from "please return JSON" in the prompt to schema-constrained generation at the API layer. Anthropic's structured outputs (stable as of October 2026) and OpenAI's Structured Outputs both mask decoding so the model can only emit output that matches your JSON schema. On the Claude API you get it two ways: output_config.format with type: "json_schema" for a JSON response, or strict: true on a tool definition for schema-exact tool inputs.

import anthropic

client = anthropic.Anthropic()

schema = {
    "type": "object",
    "properties": {
        "intent": {"type": "string", "enum": ["refund", "question", "complaint"]},
        "confidence": {"type": "number"},
        "entities": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["intent", "confidence", "entities"],
    "additionalProperties": False,
}

response = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=1024,
    tools=[{"name": "classify", "strict": True, "input_schema": schema}],
    tool_choice={"type": "tool", "name": "classify"},
    messages=[{"role": "user", "content": text}],
)
result = response.content[0].input  # matches the schema

The structural failure classes (fences, prose preamble, wrong types, missing required fields) go away because the model cannot emit them. This is the difference between validating output and making invalid output unrepresentable.

Two details bite people. First, the constrained schema is a subset of JSON Schema. Anthropic's docs list numeric constraints (minimum, maximum, multipleOf) and string length constraints as unsupported, and objects must set additionalProperties: false. Send "minimum": 0 in a raw schema and you get a 400, not a clamped number. That is why confidence above has no bounds: the bound moves to your validator. Second, a response that stopped on refusal or max_tokens can still fail the schema, so check stop_reason before you trust the payload.

Validate anyway: constraints aren't semantics

Constrained decoding guarantees the output is shaped right. It does not guarantee it's correct. confidence: 0.99 on a wrong classification is schema-valid. So the second layer is semantic validation in your own code, where the rules the decoder can't enforce live:

from typing import Literal
from pydantic import BaseModel, Field, ValidationInfo, field_validator

class Classification(BaseModel):
    intent: Literal["refund", "question", "complaint"]
    confidence: float = Field(ge=0, le=1)
    entities: list[str] = []

    @field_validator("entities")
    @classmethod
    def entities_must_appear_in_source(cls, v: list[str], info: ValidationInfo) -> list[str]:
        # drop hallucinated entities that are not in the input text
        source = info.context["source_text"].lower()
        return [e for e in v if e.lower() in source]

Field(ge=0, le=1) restores the range the provider schema couldn't carry. The validator drops entities the model invented that don't appear in the source text. No schema catches that; it's a business rule, and business rules belong in your code, not in the model's prompt. The SDK helpers (client.messages.parse() with Pydantic or Zod) do a version of this for you: they strip unsupported constraints before sending and re-check them on the response.

The repair loop, bounded

When validation still fails (usually a semantic rule, occasionally a provider or model that doesn't support strict schemas), feed the error back and retry once or twice, not forever:

from pydantic import ValidationError

def generate_validated(text: str, max_attempts: int = 2) -> Classification:
    messages = [{"role": "user", "content": text}]
    for _ in range(max_attempts):
        raw = call_model(messages)
        try:
            return Classification.model_validate(raw, context={"source_text": text})
        except ValidationError as e:
            messages.append({"role": "assistant", "content": str(raw)})
            messages.append({"role": "user",
                             "content": f"That failed validation: {e}. Fix and resend."})
    raise OutputValidationError("exhausted repair attempts")

Two rules we hold to:

• Bound the retries. An unbounded repair loop on a request the model fundamentally can't satisfy is a cost and latency bomb. Cap it, then fall back.
• Feed the specific error. "That was invalid" gets you another invalid response. "Field confidence was 1.5, must be at most 1" gets you a fix. Pydantic's ValidationError string already names the field and constraint, so pass it through verbatim.

Log every repair attempt with the error class. If one validator fires on a large share of traffic, the fix is in the prompt or the schema, not in more retries. Those logs are also the raw material for the regression cases in LLM evals in CI.

Define the fallback before you need it

When repair is exhausted, you need a defined behavior that isn't a 500. Depending on the call site:

• Route to a human queue (classification, moderation).
• Return a safe default with a degraded: true flag the caller can branch on.
• Use the last partially valid response with the failing fields nulled.

The worst outcome is an exception bubbling up to the user because nobody decided what "the model couldn't comply" should do. Write the fallback at the same time as the happy path, and test it by forcing a validator to fail.

Where to draw the trust boundary

Treat model output exactly like user input: untrusted until validated. Never let it flow directly into a SQL query, a shell command, a file path, or an outbound API call without passing your schema and semantic checks first. A schema-valid {"path": "../../etc/passwd"} is still an attack, and a prompt-injected document can steer the model into producing exactly that. The prompt injection defense patterns cover the input side; validation is the output side of the same boundary.

Constrained decoding makes the happy path clean. The validation layer is what keeps a bad day from becoming an incident.

Questions people ask

Does structured output guarantee valid JSON?

It guarantees schema-conforming JSON for responses that complete normally. Anthropic's docs note that a response ending with stop_reason of refusal or max_tokens can still fail the schema, and OpenAI returns a separate refusal field. So check the stop reason, then parse. Structured output also says nothing about whether the values are correct, which is why semantic validation stays in your code.

Why does my JSON schema with minimum and maximum return a 400 error?

Constrained decoding supports only a subset of JSON Schema. On the Claude API, numeric constraints such as minimum, maximum and multipleOf, and string length limits, are unsupported in raw schemas sent to the API. Remove them from the request schema and enforce them in Pydantic (Field(ge=0, le=1)) or Zod, or use the SDK's parse() helper, which strips and re-checks them for you.

How many times should I retry when LLM output fails validation?

One or two retries, each with the exact validation error in the follow-up message. Most fixable failures recover on the first retry when the model sees which field broke which rule. Beyond two attempts, you mostly pay latency and tokens for a request the model cannot satisfy, so route it to a fallback: a human queue, a safe default, or a degraded response.

The call we'd make

Turn on the provider's strict structured output for every call that feeds code, keep the request schema inside the supported subset, and put every range, enum-to-business-logic mapping and cross-field rule in a Pydantic or Zod model you own. Cap repair at two attempts and ship the fallback in the same pull request. The caveat: strict schemas are per-provider and per-model, so if you route across providers, your own validator is the only layer you can count on everywhere.

Sources

• Claude API docs: Structured outputs
• OpenAI API docs: Structured Outputs
• Pydantic docs: Validators