Query rewriting

Compass

Reformulating what a person asked into what the index can actually answer.

What is query rewriting?

Query rewriting is the reformulation of a user's question into one or more queries the index can answer. It resolves pronouns against the conversation and expands the abbreviations an organisation uses internally. The rewritten query is what retrieval actually receives, which makes it the most consequential step nobody inspects.

Also called: Query reformulation, Query expansion

Why the query the system searched is rarely the one asked

People do not ask questions the way an evaluation set is written. They ask what about the other one, relying on a conversation the index has no access to. They use internal acronyms that mean something different outside the company, and they bundle two questions into one sentence. Retrieval against the raw text of such a question returns passages matching its vocabulary rather than its intent.

  • The query retrieved against determines the answer more than the ranking does.
  • An unlogged rewrite makes a wrong answer impossible to diagnose afterwards.
  • Internal abbreviations collide with public ones, so expansion needs a controlled list.
  • Compound questions retrieved as one query return passages for neither part.

How query rewriting works

  • The incoming question is resolved against the session so references point somewhere.
  • Internal abbreviations are expanded against a maintained organisational vocabulary.
  • A compound question is separated into sub-queries retrieved independently.
  • Each variant is retrieved against and the result sets are merged before reranking.
  • Both the original question and every rewrite are retained with the answer.
  • Rewrites that consistently degrade retrieval are removed from the strategy.

How query rewriting is evaluated

Rewriting is evaluated by what it does to retrieval rather than by how the rewritten query reads. Compare the following against retrieval on the raw question:

  • retrieval accuracy before and after rewriting on the same question set
  • proportion of questions where rewriting degraded the result
  • coverage of the organisational abbreviation list against real queries
  • share of compound questions correctly separated into parts
  • completeness of rewrite logging against answers issued

Limitations, risks and common failure modes

  • An expansion introduces a term that pulls retrieval towards an unrelated department.
  • Pronoun resolution attaches a question to the wrong entity from earlier in the session.
  • Rewrites are applied without logging, so a complaint cannot be reproduced.
  • A rewrite strategy tuned on English queries degrades on every other language.

Related terms

RerankingA second pass that reorders retrieved passages by relevance, which is usually what separates a demonstration from production.Retrieval augmented generation (RAG)Answering from documents retrieved at the moment of the question, so the answer reflects your material rather than the model's memory.Semantic searchFinding material by what it means rather than by the words it happens to contain.Vector databaseA store that holds numerical representations of text and returns the passages closest in meaning to a query.Access-control-aware retrievalRetrieval that respects the permissions of the person asking, so an answer never exposes a document they could not open.Citation groundingAttaching every statement in an answer to the passage it came from, so a reader can check it.

All 315 terms