Intermediate10 min

WebSocket & SSE for Agents

Server-Sent Events vs WebSocket for AI agent communication. When to use each, reconnection strategies, scaling patterns, and production code for both SSE streaming and WebSocket interactive agents.

Quick Reference

→SSE (Server-Sent Events): one-way server-to-client streaming over HTTP. Perfect for LLM token streaming — simple, auto-reconnects, works through proxies.
→WebSocket: bidirectional persistent connection. Use for interactive agents where the client sends input mid-stream (e.g., cancellation, follow-ups).
→SSE reconnection is built-in: the browser auto-reconnects with the Last-Event-ID header. Send event IDs to enable resume from the last received event.
→WebSocket scaling: each connection is persistent and stateful. Use sticky sessions or connection-aware load balancers. Plan for ~10K connections per server.
→Default to SSE for LLM streaming — it is simpler, works through CDNs, and HTTP/2 multiplexes streams efficiently. Only use WebSocket when you need bidirectional communication.

SSE: Simple, One-Way Streaming

Server-Sent Events (SSE) is a simple HTTP-based protocol where the server pushes events to the client over a long-lived HTTP connection. It is one-way: the server sends, the client listens. For LLM token streaming — where the server generates tokens and the client displays them — SSE is the ideal choice. It works through CDNs, proxies, and load balancers without special configuration, and the browser's EventSource API handles reconnection automatically.

SSE streaming endpoint for agent responses with FastAPI

Client-side SSE consumption with EventSource and reconnection

WebSocket: Bidirectional Interactive Agents

WebSocket provides a persistent, bidirectional connection between client and server. Both sides can send messages at any time. Use WebSocket when the client needs to send data during an active stream — for example, cancelling a running agent, sending follow-up messages while the agent is still responding, or interactive voice agents where audio flows in both directions simultaneously.

When to Use SSE vs WebSocket

Criterion	SSE	WebSocket
Direction	Server -> Client only	Bidirectional
Protocol	HTTP (works with HTTP/2)	Upgrade from HTTP, then custom framing
Reconnection	Built-in auto-reconnect with Last-Event-ID	Manual — you must implement reconnect logic
Proxy/CDN support	Works through most proxies and CDNs natively	Requires proxy WebSocket support (most modern proxies do)
Multiplexing	HTTP/2 multiplexes multiple SSE streams on one TCP connection	Each WebSocket is a separate TCP connection
Use for agents	LLM token streaming, status updates, one-shot responses	Interactive agents, voice, real-time collaboration, cancellation
Complexity	Simple — just formatted HTTP responses	More complex — connection lifecycle, message framing, heartbeats

Sign in to read this article

This is a premium article. Sign in with your Google account to continue.