Advanced RAG & Data Architecture

62 — Hybrid Search (Dense + Sparse)

Combine lexical precision with semantic recall, then fuse candidate rankings under one authorization and evaluation contract.

Concept overview

Hybrid search combines sparse retrieval, such as term-based scoring, with dense vector retrieval. Sparse signals preserve exact identifiers and unusual terminology; dense signals connect paraphrases and related meanings. Neither channel dominates every enterprise query, so the architecture retrieves complementary evidence before selecting a final context.

The main design choice is fusion. Reciprocal rank fusion uses rank positions without assuming comparable raw scores. Weighted score fusion needs normalization and calibration because lexical and vector scores have different distributions. Filters, document versions, and deduplication must be aligned across both channels so fusion does not reintroduce unauthorized or obsolete evidence.

Why it matters in production

Proof of concept

Begin with separate lexical and dense baselines on questions containing both paraphrases and exact part numbers.

Production

Production standardizes document identity, metadata filters, fusion parameters, and candidate budgets before adding a reranker.

Enterprise scale

At scale, measure retrieval quality by query class, maintain both indexes consistently, and budget parallel lookup and fusion latency.

Architecture components

Lexical index

Finds exact names, identifiers, and distinctive terms.

Dense index

Finds passages related in meaning through query and document embeddings.

Query coordinator

Applies common constraints and dispatches bounded searches.

Fusion layer

Deduplicates document identities and combines ranks or calibrated scores.

Relevance evaluator

Measures recall, ranking quality, and downstream evidence coverage.

The architecture, mapped

Two complementary retrieval channels — Fusion happens after both retrieval paths apply the same scope and document identity rules.

flowchart LR
 Q["Question and allowed scope"] --> L["Lexical index"]
 Q --> D["Dense vector index"]
 L --> F["Rank fusion and deduplication"]
 D --> F
 F --> R["Optional cross-encoder"]
 R --> C["Evidence context"]

Runtime flow

1
Normalize the query
Preserve meaningful identifiers while handling harmless formatting differences.
2
Apply common scope
Resolve authorization and freshness constraints once for both retrieval channels.
3
Search in parallel
Fetch bounded lexical and dense candidate lists with independent timing information.
4
Fuse candidates
Combine ranks or normalized scores and retain each candidate’s source-channel evidence.
5
Rerank if needed
Score the fused shortlist against the full question and remove redundant passages.
6
Assemble context
Select cited evidence within a token budget and record retrieval diagnostics for evaluation.

Finding a replacement component

Industrial distribution · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

A distributor assistant searches catalogs and service notes for replacement components.

The problem

Exact part numbers retrieve well lexically, while descriptions such as heat-resistant valve require semantic matching.

Agent requirement

Find compatible evidence for mixed queries without losing rare identifiers.

Architecture

Catalog records and service passages share stable IDs across lexical and dense indexes.

Runtime

A query containing a legacy part number and a functional description runs through both channels.
Fusion preserves the exact-match manual and adds semantically relevant compatibility notes.

Failure scenario

If one index is behind the source revision, candidates from that revision are excluded or marked incomplete before answering.

Operational impact

Evaluate exact-identifier recall and paraphrase recall separately; aggregate relevance can conceal a serious regression in either workload.

Architecture tradeoffs

Decision	Benefit	Cost
Rank fusion	Avoids raw-score comparability assumptions.	Ignores some magnitude information within each channel.
Weighted fusion	Allows query-specific channel emphasis.	Requires calibration and drift monitoring.
Parallel retrieval	Limits serial latency.	Consumes two index paths per request.

Failure scenarios & recovery

One channel dominates numerically

Detect: Nearly all final results come from one score scale.

Recover: Use rank fusion or normalize with evaluated calibration.

Duplicate chunks crowd the context

Detect: Several hits refer to the same passage or revision.

Recover: Deduplicate by stable evidence identity before final selection.

Common architecture mistakes

Adding unnormalized scores

Treat fusion as a calibrated ranking decision.

Using different ACL filters per index

Resolve and apply one authoritative scope.

Measuring only answer fluency

Evaluate retrieval recall and citation support directly.

When to choose this pattern

Choose it when

Use for corpora mixing prose and exact identifiers.
Use when dense-only retrieval misses rare terms.
Use when lexical-only retrieval misses paraphrases.

Avoid it when

Avoid dual indexes without measured retrieval benefit.
Do not fuse results with incompatible permission semantics.
Avoid unlimited candidate lists that overwhelm reranking.

Architecture checklist

Create stable cross-index document IDs.
Align authorization filters.
Choose and document fusion semantics.
Evaluate identifier and paraphrase slices.
Deduplicate before context assembly.
Monitor index-version divergence.