92 — Pre-Computing Embeddings

Concept overview

Pre-computing embeddings moves document encoding out of the interactive retrieval path. Stable source chunks are embedded during ingestion or a scheduled backfill, then stored for repeated search. Query embeddings are usually still computed per request, though exact reusable queries may have their own cache. The approach improves latency only if precomputed vectors remain compatible and current.

A content-addressed design can avoid re-embedding unchanged chunks, but the cache key must include the embedding model, tokenizer or preprocessing contract, and relevant normalization settings. Identical text in two tenants may share computation only under an approved policy; their access metadata and retrieval identities still remain separate.

Why it matters in production

Proof of concept

A prototype compares request-time document encoding with an offline index build and records the resulting freshness delay.

Production

Production hashes source chunks, versions embedding configurations, and retries failed records without restarting the whole corpus.

Enterprise scale

Enterprise scale separates bulk backfills from incremental updates, estimates migration cost, and tracks source coverage and visibility before switching indexes.

Architecture components

Source manifest

Lists document revisions and the chunks expected in an embedding build.

Content hash cache

Reuses compatible vectors for unchanged approved content.

Embedding workers

Encode bounded batches with validated dimensions and normalization.

Vector artifact store

Persists vectors and lineage before or alongside index publication.

Coverage controller

Verifies that the expected source set is represented and current.

The architecture, mapped

flowchart LR
 S["Source revision manifest"] --> C["Versioned chunking"]
 C --> H["Content and model hash lookup"]
 H --> R["Reuse compatible vector"]
 H --> E["Encode missing chunk"]
 R --> V["Versioned vector artifacts"]
 E --> V
 V --> I["Search index publication"]
 I --> Q["Coverage and freshness checks"]

Runtime flow

1. 1

   Build a source snapshot

   Record the exact documents and revisions included in the build.

2. 2

   Normalize and chunk

   Apply a versioned preprocessing policy with stable source offsets.

3. 3

   Check reusable vectors

   Look up content hashes under the complete embedding configuration identity.

4. 4

   Encode missing chunks

   Batch only required work and retain per-record failure status.

5. 5

   Publish the index

   Write vectors with source and permission metadata, then verify search visibility.

6. 6

   Maintain incrementally

   Process changed or deleted chunks and rebuild in parallel when the embedding space changes.

Preparing a policy archive for search

Public administration · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

A records service exposes decades of approved policy documents through an internal assistant.

The problem

Encoding documents during queries makes response time unpredictable and repeats expensive work.

Agent requirement

Prepare searchable vectors offline while keeping new policy revisions current.

Architecture

A snapshot manifest drives a backfill, and content hashes reuse unchanged paragraphs across approved revisions.

Runtime

• The nightly update identifies changed chunks and encodes only those required.
• Coverage checks confirm that every active document revision has searchable vectors before publication.

Failure scenario

If an embedding model changes, a new index is built separately rather than mixing new vectors with the old space.

Operational impact

Measure build coverage, update lag, avoided encoding work, and query latency; precomputation transfers cost in time rather than eliminating it.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Hashing only raw text

Include model, preprocessing, and normalization settings.

Treating vector reuse as shared authorization

Keep tenant and source permissions separate.

Recomputing the entire corpus for small edits

Use stable chunk hashes and incremental manifests.

When to choose this pattern

Choose it when

• Use for stable or slowly changing retrieval corpora.
• Use when document encoding would dominate request latency.
• Use for large index builds with repeated unchanged content.

Avoid it when

• Avoid stale precomputation for rapidly changing data without updates.
• Do not mix incompatible embedding spaces.
• Avoid cross-scope reuse without an approved data policy.

Architecture checklist

• Snapshot source revisions.
• Version chunking and embedding settings.
• Use complete representation cache keys.
• Track per-record build status.
• Verify coverage and search visibility.
• Plan incremental updates and parallel migrations.

Related concepts

• 69 — Vector Database Scaling (Pinecone/Milvus)
• 75 — Chunk Overlap Tuning
• 76 — Multi-Aspect Indexing
• 77 — Real-time Vector Ingestion
• 79 — Embedding Model Fine-Tuning
• 96 — Semantic Similarity Analytics

Primary references

• Sentence Transformers documentation
• Milvus: Multi-vector hybrid search