Intermediate16 min
Structured Output
How to choose between function_calling, json_schema, and native structured output. Schema design, validation layers, failure modes, and the cost math for each strategy.
Quick Reference
- →model.with_structured_output(Schema) — same API across Anthropic, OpenAI, Gemini
- →method="json_schema" (ChatAnthropic) enables constrained decoding — use it in production
- →Pydantic BaseModel is the default: field descriptions guide the model, validators catch bad output
- →Four strategies: function_calling (default), json_mode (legacy), json_schema (strict), native SDK
- →Anthropic native: output_config.format with client.messages.parse() — GA on all Claude models
- →Schema compliance ≠ correct values — add @field_validator for semantic checks
- →Tool-calling adds ~200–500 tokens of schema overhead per request — factor into cost
- →Native structured output has first-request compilation latency; grammars are cached 24h after
When to Use Structured Output
Not every response should be structured
Structured output trades latency and tokens for type safety. For free-text generation, creative writing, and streaming chat UIs — don't use it. The overhead is real and you gain nothing if no downstream code consumes the fields.
| Use case | Approach | Why |
|---|---|---|
| Pipeline step consuming typed fields | with_structured_output() | Downstream code needs dot access and validation |
| Agent tool calling | bind_tools() — not with_structured_output() | Structured output is already handled inside tool dispatch |
| Streaming chat response | Raw text or JsonOutputParser | with_structured_output() blocks until the full response is parsed |
| Dynamic schema built at runtime | JSON Schema dict or JsonOutputParser | Pydantic class can't be constructed from a runtime config |
| Model without tool calling support | JsonOutputParser + format instructions | with_structured_output() requires tool calling under the hood |
| Streaming partial JSON objects | JsonOutputParser with streaming | Partial-parse tokens as they arrive; structured output can't do this |
The right question before adding structured output
Ask: does downstream code branch on a field value, store a field in a database, or pass a field to another API? If yes — structured output. If the output is only ever displayed to a human as-is — skip it.