★ OverviewIntermediate12 min
How to Design an Agent System
The decision framework for choosing between chains, single agents, and multi-agent systems — and how to scope your first agent correctly.
Quick Reference
- →Start with a deterministic chain; promote to an agent only when the task requires dynamic tool selection
- →Define the agent's scope with a clear system prompt, explicit tool list, and bounded iteration limit
- →Use the 'Can a human write a checklist?' test — if yes, use a chain; if no, use an agent
- →Design for observability from day one: structured logging, trace IDs, and LangSmith integration
- →Prototype with the strongest model (Claude Opus 4.6), then downgrade to Haiku/Sonnet once behavior is stable
Chain vs Agent — The Decision
The core question
Can the task be solved with a fixed sequence of steps? If yes → chain. If the agent needs to decide which tools to use and in what order at runtime → agent.
| Dimension | Chain | Agent | Multi-Agent |
|---|---|---|---|
| Control flow | Fixed, deterministic | Dynamic, LLM-decided | Multiple LLMs coordinating |
| Steps | Always the same | Varies per input | Varies per input and per agent |
| Tools | Not needed or fixed | 5-10, selected by LLM | Partitioned across agents |
| Cost | Low, predictable | Medium, variable | High, variable |
| Latency | 1 LLM call | 2-8 LLM calls | 5-20+ LLM calls |
| Debugging | Easy — linear trace | Medium — loop trace | Hard — distributed trace |
| When to use | Summarization, Q&A, RAG | Research, customer support | Complex orchestration, diverse domains |
The golden rule: start with a chain. Promote to an agent only when the task requires the LLM to choose between tools at runtime. Promote to multi-agent only when a single agent's tool set exceeds 8-10 tools or the task spans multiple domains.