Multi-Hop Retrieval

Handling questions that require combining information from multiple documents. Iterative retrieval, query decomposition into retrieval steps, and LangGraph-based multi-hop patterns.

Quick Reference

→Multi-hop: questions requiring information from 2+ documents that must be combined to form an answer
→Iterative retrieval: retrieve → reason about gaps → retrieve again with a refined query
→Query decomposition: break one complex question into sequential retrieval steps
→LangGraph enables state-driven retrieval loops with explicit control flow
→Limit hop count (2-4) to avoid latency explosion and compounding retrieval errors

When Single Retrieval Isn't Enough

Standard RAG retrieves documents once and generates an answer. But many real-world questions require combining facts from multiple documents. 'What was the total revenue impact of the Q3 product launches?' requires finding (1) which products launched in Q3, (2) the revenue data for each product. No single chunk contains all this information. Multi-hop retrieval solves this by performing multiple retrieval rounds, using the results of each round to inform the next query.

▸Comparison questions: 'How does Product A's pricing compare to Product B?' — needs data from both product pages
▸Aggregation questions: 'What's the total headcount across all engineering teams?' — needs data from each team's page
▸Reasoning chains: 'Is the company eligible for the tax credit?' — needs the eligibility criteria AND the company's financials
▸Temporal questions: 'How has the refund policy changed since 2022?' — needs multiple versions of the policy

Multi-hop failure rate

In production RAG systems, multi-hop questions have a 2-4x higher failure rate than single-hop questions. Each retrieval step has a chance of failing (wrong documents, missing context), and errors compound across hops. This is why explicit retrieval planning and quality checks at each step are critical.

Iterative Retrieval: Retrieve-Reason-Retrieve

The simplest multi-hop pattern is iterative retrieval. After the first retrieval, an LLM examines the results and decides: do I have enough information to answer, or do I need to retrieve more? If more is needed, the LLM generates a follow-up query based on what's missing. This continues until the LLM has enough context or a maximum hop count is reached.

Query Decomposition into Retrieval Steps

Instead of reactive gap analysis, you can proactively plan all retrieval steps upfront. The LLM decomposes the complex question into a sequence of sub-queries, each targeting a specific fact. This is more predictable than iterative retrieval and allows parallel execution of independent sub-queries.

Sign in to read this article

This is a premium article. Sign in with your Google account to continue.