Validating and Repairing LLM Extraction Output

Validating and repairing LLM extraction output takes two separate checks. One asks whether the response fits the schema. The other asks whether it faithfully represents the source. A production pipeline needs both, because valid JSON can carry wrong data straight through to the application.

Why schema validity alone is not enough to trust LLM extraction output

A ticket-routing extraction running on OpenAI strict assigned urgent to a significant share of tickets whose correct priority was normal or high. Its schema-validity rate was near-perfect. The schema check never fired because every priority was an allowed enum value. Under constrained decoding pressure, the model had collapsed to a safer enum default. The response was legal, but the routing decision was wrong.

This failure exposes the weakness in treating a successful parse as a successful extraction. A schema can require a priority field and restrict its possible values. It cannot tell which priority value is right for the ticket. Structural correctness and semantic correctness are different properties, even when the application receives them in the same object.

Native structured output modes have made syntax failures less common, which is a step forward. But constrained decoding and server-side schema enforcement constrain only the response’s shape. By making malformed responses rarer, they can also make wrong answers less visible. The schema is a delivery constraint, so every LLM response remains untrusted data after it passes that check.

The two distinct failure modes LLM extraction produces

Layer 1 consists of syntactic and schema violations. The response fails to parse, breaks a type constraint, omits a required field, or uses a value outside an enum. A deeply nested object might hit max_tokens before generation finishes, leaving invalid JSON. If the schema is unsupported, the API might instead reject the request before it returns an extraction that can be used. These failures are loud because they surface as parse errors, exceptions, or failed assertions.

Layer 2 consists of semantic errors. The response is structurally valid, but its values do not faithfully represent the source. An entity can appear in a document without belonging to the event being extracted, just as an allowed priority can be wrong for a ticket. Nothing about the response’s structure reveals that mistake. The pipeline reports success unless something separately inspects its meaning.

Jiahong Qian’s entity-extraction study makes this distinction measurable through cyber threat intelligence documents. The domain matters here as a setting for measuring attribution and confidence. The study found that a single misattribution sentence flipped high-confidence judgments in 95.6–100% of counterfactual targets. Confidence therefore did not protect the extraction from a change in attribution. The key asymmetry is that a schema validator can detect Layer 1 failures, while Layer 2 needs evidence the validator does not inspect.

How the structured output modes each fail, and what that costs in practice

Structured output modes can converge on near-identical schema-validity rates while imposing different failure shapes and costs in answer quality. OpenAI strict can reject a schema that falls outside its supported subset. Examples include missing additionalProperties set to false, optional fields not expressed as null unions, and unsupported keywords. A rejected request is a different engineering problem from a completed response containing the wrong value. The validity rate alone does not tell developers which problem they face.

Claude’s strict mode rejects some of the same constraints as OpenAI, including numeric bounds and string length constraints. It is more restrictive about recursive schemas and external $ref, which OpenAI’s strict mode supports. Checks that cannot be expressed in the provider’s supported schema need post-validation instead. Porting the schema therefore means more than changing the model name.

Gemini’s responseSchema adds another distinction. Server-side validation generally produces either a schema-valid response or an explicit rejection, but unsupported keywords can be silently ignored. The response can then arrive with HTTP 200 while violating constraints the developer expected to be enforced. Union types that have many members may cause the request to be explicitly rejected, as may schemas that are very large or deeply nested.

Outlines and JSONFormer constrain generation so that only grammar-legal tokens can be sampled. That gives a strong structural guarantee, but the cost in answer quality depends on the base model. Weaker models can fall back on the lowest-perplexity legal continuation even when it gives the wrong answer. When moving between these modes, answers can get worse without an obvious error, so the repair process must work within the limits of the chosen mode.

Repairing schema violations: the validate-repair-retry loop

The standard schema-repair pattern is straightforward. Validate the response, send the validation error back as diagnostic feedback, and retry. Its effectiveness depends on whether the feedback identifies something the model can actually fix. A generic failure message offers less guidance than an error that names the offending object and the constraint it broke. Entity-extraction research found that exceptions referencing offending objects reduced the attempts needed to produce a valid extraction.

This makes validation feedback part of the pipeline’s design. A malformed response is an expected event that needs a defined path back to generation.

Pydantic AI uses type hints and Pydantic models to define inputs, dependencies, and outputs, with invalid structure failing early at instantiation. Instructor uses a retry loop to run Pydantic validation and automatically sends any validation errors back to the model. Outlines takes a preventive approach through FSM logit masking, which blocks illegal tokens during generation. Guardrails AI and NVIDIA NeMo Guardrails provide open-source runtime guardrails that can enforce structural rules on the request path.

Truncation needs a different remedy. If the output is too long to fit within the token budget, clearer feedback cannot make it fit. The available fixes are chunking the task, reducing the schema, or increasing the token budget. Otherwise, the retry loop repeatedly attempts a structurally impossible request and adds cost without resolving the cause.

Catching semantic errors that schema validation cannot see

Once an output passes its schema check, validation must turn from shape to evidence. Grounding checks ask whether the extracted values have support in the source. Cross-field checks ask whether values that are linked to each other agree. Domain rules test requirements that a general schema cannot express or that the generation mode cannot enforce. These are separate inspections.

Legal extraction provides a concrete grounding pattern. Its hallucination-specific repair path compares references in extracted XML with references actually present in the source judgment. References found only in the output are marked as potential hallucinations. Qian’s study found that confidence signals can rank outputs for review but cannot verify their truth.

The DNA probe parsing pipeline described in 2026 used a three-stage validation process that included syntactic validation and semantic validation. Custom regular expressions checked that nucleotide sequence fields used only the allowed characters and patterns, and a post-processing recovery module dealt with outputs that were corrupted in a systematic way. For numeric and relational fields, the corresponding approach is to check ranges and cross-field constraints. When those checks fail, targeted re-extraction can return the erroneous field with feedback specifying expected units, ranges, or consistency rules.

The SEC EDGAR asset pipeline described in 2025 instead used a reference from outside the pipeline. Its two-step framework applied LSEG database validation followed by Retrieval-Augmented Validation to catch semantic errors that survived schema checks. Tools including DeepEval, Promptfoo, and Langfuse support testing and production evaluation at this broader content layer. But a model-generated check can inherit the extractor’s blindness, so a judge must be independent of the extractor to provide a genuine check.

The cost of these checks should match the risk. Entity names, financial figures, and legal references need grounding, while low-stakes formatting or labels with high base-rate agreement may not need it. This choice depends on which fields need evidence before the application can trust them.

How source quality and freshness determine whether validation can succeed

When content is noisy or its structure is unclear, it is harder to tell what belongs in a field. Stale content creates a subtler problem because the extracted value may faithfully match the document while no longer reflecting reality. An output-only validator cannot establish that the source is current.

MetaGraph’s meta-analysis of GenAI in financial NLP from 2022–2025 documented a shift toward modular, system-oriented methods, including retrieval-augmented designs. Retrieval grounds an answer in source documents. But relevance is only part of that protection. Retrieving an old document can give a confident answer a basis in facts that are out of date.

A freshness TTL, or time to live, gives source content an explicit period of validity. That period is specific to the topic. A stock price may be valid for seconds, a product spec for days, and an encyclopedia definition for weeks. Invalidation must also respond immediately to events such as a page returning 404, a published correction, or a known-volatile entity in the query.

Clean, current, token-efficient content is a prerequisite.

Source drift: when the extraction target changes underneath the pipeline

Staleness means the content has grown old, while source drift means its structure has changed enough to break the extraction contract. A pipeline may have schemas, prompts, and semantic checks calibrated to the source’s original form. After the page changes, the model can extract from it and still produce an answer that fits those old expectations. Output-side checks may pass while the result misrepresents the current source.

That makes source drift a validation problem within the pipeline’s architecture. The recommended pattern is change detection as an API primitive. When a change is detected, it becomes a pipeline event that starts re-validation, re-extraction, or schema review. It does not stop at a notification waiting for someone to decide whether it matters.

Tools built for this workflow need structured text diffs, change output that accounts for page structure, webhooks, and API access. Noise filtering separates useful changes from distracting ones, while history and snapshots preserve the material needed for comparison. With JSON or clean text output, the extraction layer can use the change without having to process another full page of noisy HTML.

An LLM summarization layer can then characterize the diff as a pricing update, policy change, new product, content removal, or layout change. Different categories can lead to different actions. A pricing change can start re-extraction right away, while an editorial change can go into a queue to be reviewed in a batch.

Putting the two layers together: a validation architecture for production extraction

The production sequence begins before generation. Fetch clean, current source content in LLM-ready Markdown. Apply topic-specific freshness TTLs and explicit invalidation, and detect structural source changes before they reach extraction. Then choose a structured output mode suited to the schema’s complexity and the model’s capability. Track retry count and token usage to keep repair costs in view.

Immediately after generation, run Layer 1 validation. Return precise errors that identify offending objects, and retry with the clarification needed to repair them. Route truncation failures toward chunking, schema reduction, or a larger token budget. Only a structurally valid response is ready for the next layer.

Layer 2 then checks the extracted meaning. Ground high-stakes fields in the source, test cross-field consistency, and consult external reference data where it exists. If a judge is used, keep it independent of the extractor rather than treating self-reported confidence as verification. For teams running multiple agents or extraction pipelines, an AI control plane such as Opencontroller can connect validation results to deployment decisions, ownership, and an audit trail.

Production monitoring should preserve these distinctions. Track schema-validity rate per mode, semantic pass rate per field, retry rate, and repair token cost. Also track source age at extraction time and source-change events that triggered re-extraction. Those measures describe different ways the pipeline can fail, so none substitutes for the others.

Validation is not one pass with one success flag. Schema checks establish that data can be consumed, while semantic checks establish whether it deserves to be used. Stacking these mechanisms covers failures that a single validator cannot see.