Custom Middleware

Build custom middleware with node-style hooks (before/after) for state updates and wrap-style hooks (wrap_model_call, wrap_tool_call) for retry, caching, and request mutation. Use request.override() to change the model or tools per call.

Quick Reference

→@before_model(can_jump_to=['end']) — early exit hook with jump support
→return {'jump_to': 'end'} — exit the agent loop from any hook
→request.override(model=model, tools=tools) — mutate model call in wrap_model_call
→ExtendedModelResponse(response, Command(update={...})) — write state from wrap_model_call
→abefore_model / aafter_model — async versions of all node hooks

Two Hook Styles

Middleware has two fundamentally different hook styles. Node-style hooks run at discrete points (before/after) and return state diffs. Wrap-style hooks surround the actual call and give you control over whether and how many times the underlying operation runs.

Hook	Style	When it runs
@before_agent	node	Once before the agent loop starts
@before_model	node	Before each model call
@after_model	node	After each model response
@after_agent	node	Once after the agent loop ends
@wrap_model_call	wrap	Around each model call — you call handler()
@wrap_tool_call	wrap	Around each tool call — you call handler()

Node-style vs wrap-style side by side

Agent Jumps — Early Exit

Return {'jump_to': 'end'} from any node-style hook to exit the agent loop early. Declare the allowed jump targets on the decorator with can_jump_to — this is required for the runtime to set up the graph edges.

Mutating Model Requests

Inside wrap_model_call, call request.override() to change the model, tools, or system message before passing to handler(). The original request is immutable — override returns a new instance.

Sign in to read this article

This is a premium article. Sign in with your Google account to continue.