74 — Hypothetical Document Embeddings (HyDE)

Concept overview

Hypothetical Document Embeddings, or HyDE, addresses the mismatch between a short query and the longer documents indexed by a dense retriever. A language model creates a plausible answer-like passage; an encoder embeds that hypothetical passage and uses it to search the real corpus. The generated text is a retrieval aid, not a factual source.

HyDE can help when a query lacks the vocabulary or structure found in relevant documents. It can also steer retrieval toward a fabricated assumption, especially for rare entities or ambiguous questions. Retaining the original query and comparing direct retrieval provides a useful control, while fusion can reduce dependence on one hypothetical generation.

Why it matters in production

Proof of concept

A prototype compares direct-query embeddings with HyDE on zero-shot retrieval examples and labels topic drift.

Production

Production constrains generation, excludes invented details from the final evidence set, and caps added model latency.

Enterprise scale

At scale, evaluate benefits by query class, cache safe hypothetical representations, and version the generator and embedding model together.

Architecture components

Hypothetical passage generator

Produces domain-shaped search text while preserving known query constraints.

Embedding encoder

Maps the generated passage into the same space as real documents.

Real document index

Returns actual source passages with provenance and authorization metadata.

Fusion controller

Optionally combines HyDE and direct-query candidates.

Evidence-only answer stage

Uses retrieved sources and excludes the hypothetical passage as authority.

The architecture, mapped

flowchart LR
 Q["Short technical query"] --> H["Hypothetical passage"]
 H --> E["Document-style embedding"]
 E --> V["Authorized real corpus"]
 Q --> D["Direct query retrieval"]
 V --> F["Candidate fusion"]
 D --> F
 F --> A["Answer from real sources only"]

Runtime flow

1. 1

   Assess suitability

   Choose HyDE when vocabulary mismatch is plausible and direct retrieval may be weak.

2. 2

   Generate search text

   Create a bounded hypothetical passage that retains explicit entities and constraints.

3. 3

   Embed the hypothesis

   Encode the passage with the corpus-compatible model.

4. 4

   Retrieve real evidence

   Search the authorized corpus and optionally combine direct-query results.

5. 5

   Validate alignment

   Reject candidates driven by unsupported hypothetical details or changed intent.

6. 6

   Answer from sources

   Generate a cited answer from real passages, or abstain if evidence remains absent.

Finding an unfamiliar failure mechanism

Materials science · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

An engineering assistant searches a technical library for descriptions of a surface defect expressed in informal language.

The problem

The user’s short description shares few terms with the relevant materials reports.

Agent requirement

Improve retrieval vocabulary without asserting a diagnosis from generated text.

Architecture

HyDE generates a candidate technical description and combines its retrieved passages with direct-query results.

Runtime

• The hypothetical passage introduces likely descriptive terminology.
• The answer stage compares real reports and states only mechanisms supported by those sources.

Failure scenario

If the hypothetical text invents an alloy or operating condition, constraint validation rejects that retrieval branch.

Operational impact

Measure recall improvement, hypothetical drift, and added latency; success means finding better real evidence, not producing a convincing invented document.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Treating HyDE as synthetic knowledge

Use it only to locate real evidence.

Adding unsupported entity details

Preserve unknowns and validate explicit constraints.

Applying it to every query

Enable it only on evaluated query classes.

When to choose this pattern

Choose it when

• Use for short descriptive queries with vocabulary mismatch.
• Use for zero-shot retrieval where relevance labels are scarce.
• Use when added latency yields measured recall gains.

Avoid it when

• Avoid exact identifier lookups already handled lexically.
• Do not use hypotheses as factual evidence.
• Avoid ambiguous high-impact questions without alignment checks.

Architecture checklist

• Retain the original query.
• Bound hypothetical passage length.
• Protect explicit entities and constraints.
• Use the corpus-compatible encoder.
• Exclude hypotheses from cited evidence.
• Evaluate against direct retrieval on the same questions.

Related concepts

• 62 — Hybrid Search (Dense + Sparse)
• 65 — Re-ranking (Cohere/Cross-Encoders)
• 66 — Query Deconstruction / Expansion
• 72 — Self-RAG / Self-Reflection
• 79 — Embedding Model Fine-Tuning
• 96 — Semantic Similarity Analytics

Primary references

• Precise Zero-Shot Dense Retrieval without Relevance Labels