Prompt Engineering for Consistent Structured Output

Getting consistent structured output from an LLM becomes a reliability problem the moment that output feeds a downstream system. A person reading a chat response can tolerate "here is a JSON-like object:" followed by slightly mangled formatting. A parser cannot. Code either gets valid JSON or it throws an exception, and there's no middle ground where a human shrugs and figures out what the model meant.

The failures that occur in production are mundane and specific. A model wraps its output in markdown fences when none were requested, and json.loads() chokes on the backticks. A model adds a sentence of preamble before the JSON block, something like "Sure, here's the extracted data," and that single sentence breaks the parse. Field names drift across runs: one response calls it "title," the next calls it "product_title," and a pipeline built around one name quietly stops populating the other. Optional fields appear in some responses and vanish in others with no pattern a regex can anticipate.

None of this points to a weak model. Frontier models are capable of producing exactly the output a developer wants most of the time, which is precisely the problem: "most of the time" is not a spec. A pipeline that works in a demo and fails once a week in production hasn't found a bad prompt. It needs a mechanism that makes the output shape a guarantee. That mechanism is what the rest of this piece is about.

Native structured output and the schema-prompt relationship

Every major model provider now ships a way to constrain output to a schema at the decoding level, and that single change moves enforcement out of the prompt and into the model's generation process itself. OpenAI offers Structured Outputs with strict JSON schema mode, plus strict function calling. Anthropic offers native Structured Outputs, configured through output schema settings. Across providers, the mechanism is the same: a developer supplies a JSON schema, the provider constrains what tokens the decoder is allowed to produce next, and the resulting output is valid JSON. Not valid most of the time. Valid.

For teams running local or self-hosted models, the same idea exists outside the major providers. Libraries like Outlines compile a schema into a finite-state machine and mask invalid tokens at every generation step, so the model is structurally incapable of emitting a token that would violate the schema. This is a different category of guarantee than asking a model nicely to behave. Instruction-following is probabilistic. Decoder-level constraint is not.

The practical effect on prompt length is dramatic. A prompt that used to run 400 words long, heavy with instructions like "Return ONLY a JSON array. Do NOT include any prose before or after the array. Do NOT wrap the output in markdown fences," collapses down to a system prompt of maybe three words, because a Pydantic model (or equivalent schema object) now carries the structural contract that all that prose used to carry. The schema says what shape the output takes. The prompt doesn't have to say it too.

What this shift does not do is eliminate the prompt's job. The schema can force the model to return a field called founding_year as an integer or null. It cannot tell the model where on the page to look for that value, what to do if two conflicting years appear, or how confident it should be before committing to a number over a null. That's still prompt work, and it is its own discipline, distinct from the version of prompting engineers used to do.

Schema design as the real engineering work

Once a schema is doing the job of enforcing structure, the quality of what comes out of the pipeline depends mostly on how well that schema is designed. A poorly designed schema still produces technically valid JSON. It just produces valid JSON that's useless: fields too broad to act on, missing fields that force the model to guess, enum values that don't cover the cases the pipeline actually encounters.

Field granularity is one of the first decisions that separates a schema that works from one that doesn't. A single address string forces the model to decide, run by run, what belongs inside it and in what order. Splitting that into street, city, and postal_code removes the ambiguity entirely, because each field has exactly one kind of value that belongs in it. The same logic applies to the choice between required and optional fields. Marking a field required when the source material might not actually contain that information all but guarantees a hallucinated value, because the model has to put something there. Marking it optional, with a nullable type, gives the model permission to be honest when the data simply isn't on the page.

Enum constraints do similar work for categorical fields like status, sentiment, or category. A closed enum eliminates the drift that happens when a model is free to invent its own label for something that should only ever take one of five known values. Field descriptions matter just as much as the field's type. A short description attached to a schema property functions as an instruction scoped to that one field, the same way a sentence of prompt text would, except it travels with the schema instead of living in a block of instructions the model has to parse out and apply correctly. Nesting depth also affects reliability: a deeply nested schema gives the model more places to misplace a value, so flattening structure where the data allows it tends to produce cleaner extractions.

Research out of UCLA on a system called OOPrompt backs up the broader shift this represents. That work treats prompts as structured, manipulable artifacts, decomposing intent into adjustable properties rather than a single block of linear text, and finds that this approach makes planning and iterative work faster and clearer, and handles ambiguous, exploratory tasks better than prompting written as a paragraph. Writing a JSON schema for extraction is a lot closer to writing a typed function signature than to writing a paragraph of instructions. A function signature tells you what goes in and what comes out, with no room for a caller to guess. A well-built schema does the same thing for a model.

What the prompt does in a schema-first world

With structure handled, the prompt's job narrows to intent, source handling, and ambiguity resolution, and getting those three things right is what separates a correct extracted value from one that is merely correctly shaped.

The prompt has to say what content to extract and from where. A page might have a price in a hero banner and a different price in the fine print, and the schema has no way of knowing which one the pipeline actually wants. The prompt has to state what the model should do when a field's value is absent, ambiguous, or contradicted by two different parts of the source. An instruction as simple as "if the value is not present in the source, return null" does more work than most developers expect, because it closes off the model's instinct to produce a plausible-sounding guess when it finds nothing. The prompt also has to set priority rules for cases where more than one candidate value exists for a single field, since the schema will accept whichever one the model picks without knowing or caring whether it was the right one.

Structuring the prompt also means handling the page content itself with some discipline. Wrapping the source text in delimiters, XML tags work well for this, draws a clear boundary between the developer's instructions and the content being processed. That boundary does two things at once: it helps the model understand which text to read as data versus which text to read as instructions, and it reduces the risk that something embedded in scraped content gets misread as a command. For pipelines ingesting arbitrary web pages, that second benefit is not optional.

A clean structure for the prompt puts the schema contract, the model's persona, and any persistent rules in the system prompt, and reserves the user turn for the actual page content and whatever instructions are specific to that request. A short checklist at the end of the prompt, something the model checks its own draft output against before finalizing, catches a category of mistakes that neither the schema nor a well-worded instruction set will catch on its own.

Few-shot examples as schema validators, not style guides

Few-shot examples used to exist mostly to show a model what tone or format a response should take. In a schema-first pipeline, with the schema already handling format, examples earn their place by showing the model how to handle the cases a schema alone can't resolve.

The clearest use case is the null case. A page genuinely doesn't contain the value a field asks for, and the correct response is a null, not a confident guess dressed up to look like a real answer. One well-placed example showing exactly that, a field left null because the source simply didn't have the information, prevents a kind of hallucination that no amount of schema design can catch on its own, because the schema permits a value in that field. It just doesn't know the model shouldn't invent one.

The second use case is ambiguity. Two prices appear on the same page. A name could belong to a person or a company. An example that walks through how to resolve that kind of conflict gives the model a pattern to follow instead of leaving it to decide case by case with no guidance. The most useful examples for this purpose don't come from imagination. They come from real failures already observed in production: the specific pages where the model returned something plausible but wrong are exactly the cases worth turning into a demonstration, because they represent the failure mode most likely to recur.

Chain-of-thought and self-critique for high-stakes extraction

Not every technique from the earlier era of prompt engineering survived the move to schema-first pipelines, and chain-of-thought is a good example of one that needs selective use, applied only when the task calls for it. Modern reasoning models already perform chain-of-thought internally, regardless of the task. Asking for it explicitly in a straightforward extraction task adds tokens to the request without a meaningful gain in accuracy to show for it.

Explicit chain-of-thought earns its cost in extraction tasks that require real inference. Multi-hop inference is one example: a field value isn't stated outright on the page but has to be derived, such as working out a company's founding year from a line like "celebrating 12 years" in an anniversary announcement. Conflict resolution is another: two sources on the same page disagree, and the model has to reason about which one is authoritative before it can commit to a value. Nested entity extraction is a third: a page describes multiple products or people, and the model has to figure out which attributes belong to which entity before it can populate the schema correctly. In each of these cases, giving the model room to reason explicitly, rather than silently, produces a more reliable result.

Self-critique works as a two-pass pattern that complements this. The model generates the extraction first, then reviews its own output against a short checklist before finalizing it. That second pass catches mistakes that would otherwise pass silently, like a field populated with a value pulled from the wrong entity on a page describing several similar ones. The checklist for that review should stay scoped tightly to the schema: does every required field have a value or an explicit null, does any value look invented rather than pulled from the source, do enum fields stay within the set of allowed values. A second pass costs more tokens and more time than a single pass. Weighed against a wrong value propagating silently into a downstream system, that cost is the cheaper option.

Tool-calling as the schema-first pattern for agentic extraction

The same schema-first logic that fixes single-turn extraction also fixes the control-flow problems that used to plague agentic pipelines. When a model's tools are defined as JSON schemas, the model returns a structured tool call instead of prose that code then has to parse, and that removes an entire category of fragility from agent loops.

The older approach to agents relied on something like a ReAct scratchpad, where the model wrote out its reasoning and an "Action:" line that code parsed with regex. Control flow lived inside the prompt in that setup, and changing a single word in the model's phrasing could silently break the parse. The current approach defines each tool with a JSON schema: a name, a description, and parameters with their own types and descriptions. The model returns a structured tool call matching that schema, code dispatches the call, and the result comes back as a tool message the model can read and act on next.

No parsing, no regex hunting for an "Action:" prefix, no fragile string matching standing between the model's intent and the code executing it. The loop itself becomes simple, almost boring, and the schema carries the interface. What used to be prompt engineering time spent wording an agent's instructions carefully becomes schema design time spent deciding which tools to expose, how to name their parameters, and what their descriptions say, since those descriptions are now the instructions the model actually acts on.

Applied to web extraction, a tool defined as something like extract_structured_data(url: string, schema: object) gives an agent a typed, auditable way to pull data from a live page. The agent decides when to call it based on what it's trying to accomplish. The prompt doesn't have to choreograph that decision step by step.

Applying schema-first extraction to live web data pipelines

Web content is the hardest input a structured extraction pipeline will ever face, because no two pages present the same information the same way, pages change layout without warning, and scraped text can contain material crafted to manipulate a model's instructions. A price might be a clearly labeled field on one site and a buried sentence fragment on another, and a schema built for one won't automatically generalize to the other without a model doing real reading in between.

This is exactly where schema-first extraction has an advantage over selector-based scraping. A CSS selector targets a specific location on a page, and that location can vanish the next time the site redesigns. A schema targets a piece of information semantically, regardless of where it sits on the page, and a model reading the content maps what it finds onto the right field rather than hunting for a path in the DOM that may no longer exist.

An analysis found that cleaning scraped pages down to clean Markdown, stripping all of that out, can account for 40% of the token count in raw scraped pages. Giving the model clean Markdown instead of raw HTML keeps the content the schema fields need while cutting a meaningful share of wasted tokens.

A tiered extraction architecture balances cost against reliability across this range of page types. Fast selectors handle well-structured, stable pages where layout isn't likely to shift. LLM extraction against the schema steps in as a fallback for irregular or prose-heavy pages where no fixed selector will hold up. Vision extraction serves as a last resort for pages where the content lives inside images or renders without an accessible DOM. And throughout all three tiers, wrapping scraped content in delimiters, XML tags or triple backticks, keeps the developer's instructions separated from the page content itself, so the model knows which text to extract from and which text to simply trust as instruction.

Schema versioning and change detection as ongoing pipeline maintenance

A schema-first pipeline holds up far better against a changing web than a selector-based one does, but the schema itself isn't a one-time artifact. It needs versioning, and the pipeline around it needs some way of noticing when a source has changed shape.

Three things break even a well-designed schema over time. A source adds a new field the schema was never built to capture, and that data disappears silently with no error raised anywhere. A source restructures how it presents an existing field, and the model may start populating the wrong schema field with a value that's technically correct but in the wrong place. A source removes a field the schema marks as required, and extraction starts returning nulls, or worse, starts hallucinating values to satisfy a requirement that no longer has real data behind it.

Treating the schema as code addresses the first problem directly. Version control, a changelog, and downstream systems pinned to a specific schema version rather than consuming whatever the latest definition happens to be, all keep a schema change from becoming a surprise somewhere downstream. Keeping field keys stable across versions serves the same purpose API versioning does: a downstream consumer built around a field called price shouldn't have to change its code every time the schema gets a minor update elsewhere.

Monitoring field population rates catches the second and third problems before they do real damage. A required field that suddenly starts returning null across a large share of pages is a signal that a source has changed layout, not that the model got worse. When that kind of anomaly occurs, the pipeline can trigger a re-extraction and validate the new result against the schema, with the schema itself acting as the acceptance test for whether the re-run succeeded.

The core advantage schema-first pipelines hold onto through all of this is visibility. A null sitting in a typed field is something a monitoring system can see and flag. A CSS selector that quietly returns nothing looks identical to one that was never triggered. Schema-first pipelines fail in ways that show up on a dashboard. Selector-based ones tend to fail in ways nobody notices until a downstream report comes back empty.