Skip to content
Gradient Notes logoGradient Notes
LLM Engineering

Structured outputs without tears

Getting reliable JSON out of a language model is a solved problem — if you combine schemas, validation and a sensible retry policy.

Maya Okafor2 min read

Close-up of copper traces on a green printed circuit board
Peter Shanks from Hobart, Australia · CC BY 2.0

Early on, getting JSON from a model meant begging in the prompt and parsing with a regular expression. Today most providers support constrained decoding or "structured output" modes that guarantee syntactically valid JSON matching a schema. That solves half the problem. The other half is making sure the values are right.

Define the schema once

Keep a single schema in code and derive everything from it: the JSON Schema you send to the provider, the TypeScript type your application uses, and the runtime validator. With Zod that looks like this:

schema.tstypescript
import { z } from 'zod'

export const Invoice = z.object({
  vendor: z.string().min(1),
  invoiceNumber: z.string(),
  issuedOn: z.string().date(),
  currency: z.enum(['USD', 'EUR', 'GBP', 'AED']),
  lines: z.array(z.object({
    description: z.string(),
    quantity: z.number().positive(),
    unitPrice: z.number().nonnegative(),
  })).min(1),
  total: z.number().nonnegative(),
})

export type Invoice = z.infer<typeof Invoice>

Validate semantics, not just shape

A schema guarantees that total is a number. It does not guarantee that it equals the sum of the line items. Add a second layer of business checks after parsing: totals add up, dates are not in the future, the currency matches the vendor's country.

When a check fails, send the model its own output plus a short, specific error ("total 1,240.00 does not equal the sum of lines 1,140.00") and ask it to correct it. One retry fixes the large majority of cases; more than two retries rarely helps.

Make fields optional on purpose

If the model must fill every field, it will invent values for missing data. Mark fields as nullable when the source might not contain them, and tell the model that null is a perfectly good answer.

Keep humans in the loop for low confidence

For high-stakes extraction, ask for a confidence field per value or compare two independent extractions. Disagreements go to a review queue. It is much cheaper to review 5% of documents than to clean up after silently wrong data.

Keep reading