Advanced11 min

Classification & Routing Patterns

Intent-based routing, semantic routing, multi-level classification, and fallback strategies for directing queries to specialized handlers with confidence thresholds.

Quick Reference

→Intent classification: classify user query into categories, route to specialized handler for each category
→Semantic routing: embed the query and compare to route embeddings — the closest match wins
→Multi-level classification: coarse category first (e.g., 'support'), then fine-grained (e.g., 'billing/refund/technical')
→Confidence thresholds: only route when classification confidence exceeds a threshold — fall back to a general handler otherwise
→LLM-based routing is more flexible; embedding-based routing is faster and cheaper — use both in a tiered approach

Why Classification and Routing Matter

Not every query should hit the same agent or model. A simple FAQ question doesn't need a $0.10 Sonnet call with 5 tools — a cached response or a Haiku call is cheaper and faster. Routing classifies incoming queries and sends them to the right handler, optimizing for cost, latency, and accuracy simultaneously.

Routing Strategy	Speed	Cost	Accuracy	Best For
Keyword matching	< 1ms	Free	Low — brittle rules	Known exact patterns (commands, codes)
Embedding similarity	5-20ms	~$0.0001/query	Medium — semantic matching	Routing to templates, FAQ matching
LLM classification	500-2000ms	$0.001-0.01/query	High — understands nuance	Complex routing, ambiguous queries
Multi-level (embedding → LLM)	20-2100ms	Varies	Highest	Production systems with diverse traffic

Intent-Based Routing

Intent classification identifies what the user wants to do (their intent) and routes to a specialized handler. Each handler has its own system prompt, tools, and possibly its own model — optimized for that specific task.

Semantic Routing with Embeddings

For high-throughput systems where LLM classification is too slow or expensive, embedding-based semantic routing provides a fast alternative. Pre-compute embeddings for each route's description, then compare incoming query embeddings to find the closest match.

Sign in to read this article

This is a premium article. Sign in with your Google account to continue.