68 — Time-Weighted Vector Retrieval

Concept overview

Time-weighted retrieval adjusts evidence ranking using temporal information. A recent incident update may matter more than an older operational note, while a foundational specification may remain authoritative for years. The correct time signal depends on the question: publication date, last modification, event occurrence, and effective validity are not interchangeable.

A common architecture combines a relevance score with a calibrated freshness function after applying hard validity constraints. Freshness is a preference; superseded or not-yet-effective policy status is often an eligibility rule. Time decay should be tuned by document class and query intent rather than applied uniformly to the entire corpus.

Why it matters in production

Proof of concept

A prototype compares relevance-only results with time-aware ranking on explicitly time-sensitive questions.

Production

Production normalizes timestamps, preserves source timezone and validity, and distinguishes live status from historical queries.

Enterprise scale

At scale, handle late-arriving events, clock errors, source-specific freshness contracts, and evaluation across both current and historical evidence needs.

Architecture components

Temporal metadata parser

Extracts publication, event, modification, and effective dates with provenance.

Query time resolver

Determines whether the user asks about now, a historical point, or a period.

Eligibility filter

Excludes evidence outside required validity or authorization constraints.

Ranking combiner

Balances semantic relevance and document-class-specific recency.

Freshness monitor

Detects delayed ingestion and source update gaps.

The architecture, mapped

flowchart LR
 Q["Question and reference time"] --> V["Validity filter"]
 V --> R["Semantic candidates"]
 R --> T["Class-specific time weighting"]
 M["Event and publication metadata"] --> T
 T --> C["Revision conflict check"]
 C --> A["Answer with evidence dates"]

Runtime flow

1. 1

   Resolve temporal intent

   Identify the requested reference time and clarify internally whether current or historical evidence is needed.

2. 2

   Filter validity

   Exclude superseded or future-effective records where the task requires active guidance.

3. 3

   Retrieve semantically

   Collect relevant candidates without allowing recency to substitute for topic relevance.

4. 4

   Apply time weighting

   Use the appropriate timestamp and calibrated decay for each evidence class.

5. 5

   Check conflicting revisions

   Prefer authoritative active versions while retaining historical context when requested.

6. 6

   Expose freshness

   Attach source dates and any ingestion lag to the answer’s evidence.

Summarizing an evolving incident

Cybersecurity operations · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

An incident assistant summarizes an active service compromise from investigation notes and status updates.

The problem

An older high-similarity note describes a hypothesis that later evidence disproved.

Agent requirement

Favor current supported findings while retaining the timeline of changed understanding.

Architecture

Incident documents carry event time, publication time, and supersession links.

Runtime

• The query asks for current containment status.
• The retriever prioritizes the latest authoritative status while linking earlier hypotheses as historical context.

Failure scenario

A late-arriving note with an old event timestamp does not automatically replace newer operational status.

Operational impact

Track stale-evidence selections, source lag, and historical-query accuracy; freshness improvements should not erase useful older evidence.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Using file modification as event time

Store the different temporal meanings separately.

Applying one decay rate to every source

Calibrate by document class and task.

Equating new with authoritative

Preserve source authority and validity rules.

When to choose this pattern

Choose it when

• Use for incidents, news, inventory, and changing policies.
• Use when users ask time-qualified questions.
• Use when stale evidence is a measured failure mode.

Avoid it when

• Avoid aggressive decay for stable technical standards.
• Do not infer freshness from missing timestamps.
• Avoid recency weighting as a replacement for version control.

Architecture checklist

• Define timestamp semantics.
• Normalize timezones consistently.
• Validate clock and future-date anomalies.
• Distinguish validity from ranking preference.
• Test historical as well as current queries.
• Monitor source-to-index lag.

Related concepts

• 61 — Semantic Caching (Redis/GPTCache)
• 63 — Metadata Pre-Filtering
• 67 — Graph RAG / Knowledge Graphs
• 69 — Vector Database Scaling (Pinecone/Milvus)
• 77 — Real-time Vector Ingestion
• 93 — Model Versioning and A/B Testing

Primary references

• Milvus: Filtered search
• Microsoft GraphRAG: Indexing overview