Agent Structured Output
Before wiring up ProviderStrategy or ToolStrategy, you need to know when structured output will hurt you — streaming breaks, retries compound cost, and over-constrained schemas hallucinate values. This article covers the decision, the cost math, two failure modes most tutorials skip, and the schema patterns that cut retry rates.
Quick Reference
- →result['structured_response'] — where the validated output lives after agent.invoke()
- →response_format=MySchema — shorthand, auto-selects ProviderStrategy then falls back to ToolStrategy
- →ProviderStrategy(schema, strict=True) — provider enforces schema via grammar-constrained decoding
- →ToolStrategy(schema, handle_errors=True) — tool-call approach; each retry is a full LLM call
- →ToolStrategy(Union[SchemaA, SchemaB]) — let the model choose which schema fits the input
- →ToolStrategy has no built-in retry cap — your handle_errors callable is your budget gate
- →stream_mode='messages' breaks with both strategies — stream steps, capture result at the end
- →Flat schemas with short Field descriptions retry less than deep nested schemas
Should You Use Structured Output at All?
All paths converge on semantic validation — constrained decoding handles syntax only
Structured output is not the right tool for every extraction task. Three cases where it makes things worse: (1) Exploratory chat — if the user expects prose, forcing a schema produces robotic responses that feel wrong even when technically valid. (2) Streaming-first UIs — neither ProviderStrategy nor ToolStrategy delivers token-by-token streaming of the final response. If your UI displays text as it arrives, structured output breaks the experience. (3) Over-constrained schemas — if you require a confidence_score field but the model has no signal for it, it will hallucinate a number. A required field with no grounding is worse than no field at all.
| Approach | Reliability | Streaming | Cost overhead |
|---|---|---|---|
| Free text + regex | Low — brittle on variation | Full | Zero |
| with_structured_output (LCEL) | Medium — single attempt, no retry | Partial | Minimal |
| ProviderStrategy via create_agent | High — provider-enforced | None on final output | Minimal |
| ToolStrategy via create_agent | High — with retries | None on final output | Each retry = 1 LLM call |