LangChain/Core Concepts
Beginner14 min

Chat Models & Providers

How to choose a provider, wire it up, configure it, and handle the failures that happen in production. LangChain's ChatModel interface gives you one API for all providers — but the decisions around which provider, what configuration, and how to recover from errors are yours to make.

Quick Reference

  • →One provider package per vendor: langchain-anthropic, langchain-openai, langchain-google-genai, langchain-aws
  • →All providers share: .invoke(), .stream(), .batch(), .ainvoke(), .astream()
  • →Provider cost as of April 2026: Gemini 2.5 Flash ($0.30/$2.50/1M) → GPT-5.4-mini ($0.75/$4.50/1M) → Haiku 4.5 ($1/$5/1M) → GPT-5.4 ($2.50/$15/1M) → Sonnet 4.6 ($3/$15/1M) → Opus 4.7 ($5/$25/1M)
  • →Model profiles (v1.1): use .profile.max_input_tokens, .profile.tool_calling, .profile.image_inputs — not context_window/supports_tool_use
  • →Anthropic's max_tokens defaults to 1024 — always set it explicitly or responses will be silently truncated
  • →Use .with_fallbacks([backup_model]) for resilience; use model-retry middleware for transient rate limits
  • →Never hardcode provider logic in your chain — pick the model via config/env, not via if/else in business logic

When NOT to Use LangChain's Chat Model Layer

Skip the abstraction when you don't need it

If you're using one provider permanently, building a thin API proxy, or running in a latency-critical path (edge functions, sub-100ms SLA), use the provider SDK directly. LangChain's abstraction adds ~30–80ms cold start overhead per Lambda/container due to import chain depth. You only pay this cost once per warm instance, but in serverless architectures that run cold often, it's measurable.

  • ▸Use LangChain ChatModel when: you have multiple providers, you're building chains/agents that need composability, or you want built-in retry/streaming middleware
  • ▸Skip LangChain when: single provider, latency is P99 critical, team is already productive on the provider's native SDK, or you only need one or two API calls
  • ▸Using langchain-anthropic directly (no langchain-core) is valid — the provider packages work standalone if you need just the ChatModel class without the full framework