70 — Dynamic Prompt Assembly

Concept overview

Dynamic prompt assembly constructs a request from several sources at runtime rather than concatenating a fixed template with a user message. The assembler selects relevant instructions, tool definitions, conversation state, retrieved evidence, and output constraints. Its architectural role is to preserve priority and provenance while adapting context to the task.

Assembly is a trust boundary. Retrieved documents and tool results are data even when they contain imperative language. Policy instructions must remain application-controlled, and user-supplied text must not be promoted into a privileged instruction slot. A manifest of selected segments makes requests reproducible without storing unrestricted sensitive payloads in general logs.

Why it matters in production

Proof of concept

A prototype separates instruction templates from user and retrieval data and records prompt versions.

Production

Production applies authorization before inclusion, token budgeting, structured delimiters, and deterministic segment ordering.

Enterprise scale

Enterprise scale adds policy-version rollout, prompt manifests, locale-specific templates, and evaluation of assembly changes across task and data classes.

Architecture components

Template registry

Stores reviewed instruction and output-contract versions.

Context selector

Chooses task-relevant history and authorized source evidence.

Policy resolver

Determines applicable tenant, data, and tool restrictions.

Token budgeter

Reserves room for output and allocates context by priority.

Prompt manifest

Records segment identities, versions, trust levels, and exclusion reasons.

The architecture, mapped

flowchart TD
 P["Reviewed policy templates"] --> A["Prompt assembler"]
 T["Task and output contract"] --> A
 E["Authorized evidence"] --> B["Data segment boundary"]
 H["Selected conversation state"] --> B
 B --> A
 A --> K["Token and schema validation"]
 K --> M["Model request and manifest"]

Runtime flow

1. 1

   Resolve task contract

   Determine requested output, model capability, and applicable policy.

2. 2

   Collect candidate segments

   Load instructions, conversation state, evidence, and tool schemas with provenance.

3. 3

   Enforce trust and access

   Reject unauthorized evidence and keep untrusted content in clearly bounded data fields.

4. 4

   Allocate tokens

   Reserve output capacity and choose segments according to task value and required constraints.

5. 5

   Serialize deterministically

   Render stable instructions first and changing task data in consistent locations.

6. 6

   Validate and trace

   Check size and schema, then record the manifest and evaluate the response against its contract.

Answering region-specific leave questions

Human resources · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

An employee assistant uses company policies, regional addenda, and the employee’s selected locale.

The problem

A single oversized prompt mixes irrelevant regions and sometimes includes obsolete policy fragments.

Agent requirement

Assemble only applicable evidence while preserving access limits and a consistent answer format.

Architecture

The policy resolver selects the region, the retriever supplies authorized active documents, and the assembler creates a versioned manifest.

Runtime

• The task requests a leave-policy explanation for one region.
• The assembler includes the active regional addendum and reserves room for citations and uncertainty.

Failure scenario

If mandatory policy text cannot fit, assembly fails explicitly or chooses a supported larger-context route instead of silently dropping it.

Operational impact

Track assembly failures, excluded evidence, token use, and policy-citation accuracy; reproducible manifests help explain behavior changes.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Concatenating everything into one string

Use typed segments with explicit trust and provenance.

Dropping instructions to meet token limits

Reserve mandatory policy capacity before optional evidence.

Logging complete prompts indiscriminately

Store restricted manifests and governed payload references.

When to choose this pattern

Choose it when

• Use for tasks with changing evidence and policy context.
• Use when several tools or output schemas are selected dynamically.
• Use when prompt reproducibility matters operationally.

Avoid it when

• Avoid elaborate assembly for a stable tiny classifier.
• Do not let model output set privileged instruction content.
• Do not include data merely because capacity remains.

Architecture checklist

• Version instruction templates.
• Label segment trust and provenance.
• Authorize evidence before inclusion.
• Reserve output and mandatory-policy tokens.
• Record assembly manifests.
• Evaluate prompt changes across representative task slices.

Related concepts

• 19 — Context Condensation
• 24 — Prompt Injection Mitigation
• 28 — Deterministic Data Grounding
• 63 — Metadata Pre-Filtering
• 71 — Context Window Optimization
• 99 — Context Caching (Anthropic/OpenAI Feature)

Primary references

• Lost in the Middle: How Language Models Use Long Contexts
• OpenAI API: Prompt caching