The Query Expansion article closed on a genuine distinction worth honoring precisely. This article covers that distinct technique: query rewriting.
The simple definition
Query rewriting restructures or replaces a user’s original query with a clearer, more search-friendly version before retrieval runs, rather than simply adding extra terms alongside the original. Recall from the Query Expansion article’s core distinction: expansion adds to the query; rewriting transforms it.
A vague, conversational question like “what about the electronics one” might get rewritten into a genuinely searchable query like “what is the return policy for electronics” before retrieval ever runs.
Why real user questions often aren’t naturally search-friendly
Recall from the Prompt article’s discussion of context windows and ongoing conversations — a real user’s question in a chat interface often depends heavily on earlier conversation history, using pronouns and shorthand that only make sense in that specific context.
“What about the electronics one” only means anything if you already know the earlier conversation was about return policies. Feeding this vague fragment directly into retrieval, without any rewriting, would likely fail to match the actually relevant chunk, since it lacks the specific, explicit terms retrieval needs to work well.
flowchart LR
A["Vague, conversational query: 'what about the electronics one'"] --> B[Query Rewriting: incorporate conversation context]
B --> C["Rewritten query: 'what is the return policy for electronics'"]
C --> D[Retrieval runs on this clearer, self-contained version]ANALOGY vs. TECHNICAL REALITY
Analogy: Think of a skilled research librarian who, hearing a patron ask a vague, context-dependent question like “does it work for that too,” first translates it into a precise, self-contained question — “does this specific software support PDF export” — before actually going to search the archive, rather than trying to search using the vague version directly.
Where this breaks down: A librarian’s translation draws on genuine conversational understanding built over the whole interaction. Query rewriting in a real RAG system is typically performed by a language model itself, prompted specifically to reformulate the query using the conversation history as context — the same underlying next-token-prediction mechanism covered throughout the Language Models phase, applied here as a dedicated preprocessing step before retrieval, rather than genuine conversational comprehension.
How this actually gets done, step by step
This is worth being concrete about, since it’s a real, common architectural pattern in production RAG systems.
Before the main retrieval step runs, a language model call — often a smaller, faster model than the one generating the final answer — takes the conversation history and the user’s latest message, and produces a rewritten, self-contained query specifically optimized for search: resolving pronouns, filling in implied context, and often removing conversational filler entirely.
A concrete example, layered
For a simple beginner example: in a multi-turn conversation where a user first asks “tell me about your premium plan” and then follows up with “how much does it cost,” query rewriting transforms that follow-up into the self-contained “how much does the premium plan cost” before retrieval runs, since the original fragment alone shares no useful keywords with a pricing document.
For a production example: LangChain and LlamaIndex, referenced throughout this phase, both provide built-in query rewriting components specifically for multi-turn conversational RAG applications, reflecting how standard this preprocessing step has become in real, deployed chatbot products that need to handle natural, contextual conversation rather than isolated, one-off questions.
Why this step genuinely matters for real conversational products
It’s worth connecting this directly back to the Retrieval article’s core finding: retrieval quality determines everything downstream. A perfectly capable retriever will still fail if it’s given a genuinely unsearchable, context-dependent fragment as its input — query rewriting exists specifically to ensure retrieval always receives a query it can actually work with well, regardless of how naturally or vaguely the user actually phrased their question in a real, ongoing conversation.
Rewrite a conversational question
Earlier message: “I bought the headphones last Tuesday.”
New question: “Can I return them after opening the box?”
Standalone search query:
“Return policy for opened headphones purchased last Tuesday”The rewritten query resolves them using conversation history and adds the product category. It should not invent a purchase location or reason for return that the user never supplied.
Query rewriting is useful for vague follow-up questions, spelling normalization, acronyms, and translating conversational language into searchable terms. Applications should log both original and rewritten queries so retrieval failures can be diagnosed.
Query rewriting in a real search response
Google’s Search response can expose a rewrittenQuery and separately report filters extracted from natural language. For “show waterproof cameras under ₹50,000”, the system can search a rewritten product query while turning the price requirement into a structured filter.
This is useful because the retriever receives a clearer search instruction without losing the user’s constraint. See Google’s SearchResponse reference.
Common misconception
A frequent beginner assumption: that query rewriting is only useful for fixing typos or grammatical errors. As this article’s conversational example demonstrated, its real, primary value is resolving conversational context and implied meaning — turning a fragment that only makes sense within a specific conversation history into a clear, standalone query retrieval can actually use, a much more substantial transformation than simple spelling correction.
Where this fits in what comes next
You now understand both major ways of transforming a query before search — expansion (adding related terms) and rewriting (restructuring the query itself). The next article, Context Retrieval, covers a related, complementary concern — making sure a retrieved chunk carries enough of its own surrounding context to be genuinely useful once it’s found.
In one sentence
Query rewriting restructures or replaces a user’s original query — commonly using a language model to resolve conversational context and implied meaning — into a clearer, self-contained version before retrieval runs, a standard, necessary preprocessing step for any real conversational RAG product handling natural, multi-turn dialogue.
Related Terms
- Author
- TechByteByByte Editorial Team
- Reviewed by
- TechByteByByte Admin
- Published
- Last reviewed