Agentic Frameworks & Coordination

5 — Dynamic Routing

Select the appropriate agent or workflow from request intent, capability, authorization, and uncertainty while keeping routing decisions observable and reversible.

Concept overview

Dynamic routing maps an incoming request to an execution path using information available at runtime. The destination may be a specialist agent, a deterministic workflow, a clarification step, or a human queue. Routing should separate eligibility from preference: policy determines which destinations are allowed, then quality and operational constraints determine which eligible destination is best.

A classifier's confidence is not a calibrated probability unless validated as one. Preserve an ambiguity route and evaluate routing against downstream task outcomes. A route that looks semantically plausible can still be wrong if the selected worker lacks the necessary tool, geography, or data entitlement.

Why it matters in production

Proof of concept

In a prototype, hand-labeled examples and a small rule table reveal the main intent boundaries before an LLM router is introduced.

Production

Production requires a versioned capability registry, authorization filtering, calibrated acceptance thresholds, and a clear fallback for mixed intent. Log route reasons without exposing sensitive inputs.

Enterprise scale

At scale, routing also accounts for regional availability, tenant policy, queue pressure, and cost. Avoid rapid route oscillation by using stable decision windows and recording the chosen policy version.

Architecture components

Request normalizer

Extracts language, product, tenant, channel, and explicit requested action.

Eligibility filter

Removes destinations that lack permissions, residency compatibility, or required capabilities.

Intent router

Ranks eligible destinations and identifies ambiguity or unsupported requests.

Capability registry

Publishes versioned worker contracts, supported intents, and operational availability.

Outcome feedback

Compares chosen routes with corrections, escalations, and actual task completion.

The architecture, mapped

Policy-constrained service routing — Policy reduces the candidate set before intent classification; uncertain requests retain an explicit destination.

flowchart LR
 Q["Service request"] --> E["Eligibility filter"]
 P["Identity and tenant policy"] --> E
 E --> R["Intent router"]
 R --> B["Billing workflow"]
 R --> N["Network diagnostics"]
 R --> C["Clarification queue"]
 B --> O["Outcome evaluation"]
 N --> O
 C --> O
 O --> D["Reviewed routing dataset"]

Runtime flow

1
Normalize request
Preserve the original request while extracting trusted identity and business context separately.
2
Filter destinations
Apply tenant policy and capability requirements before any probabilistic classification result is considered.
3
Classify intent
Produce a structured route proposal with relevant evidence and an ambiguity indicator.
4
Validate selection
Confirm destination health and required tools; reject nonexistent or unauthorized route identifiers.
5
Dispatch context
Send the minimum required request packet with route and policy versions attached.
6
Observe outcome
Record corrections and unresolved cases for evaluation rather than automatically retraining on every user interaction.

Routing enterprise support requests

Telecommunications · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

A telecom provider receives requests combining account charges, connectivity outages, and service changes across several countries.

The problem

Keyword routing sends an outage described as a billing problem to a team that cannot inspect network telemetry.

Agent requirement

The assistant must identify the required capability and respect country-specific service access without losing mixed requests.

Architecture

A policy filter constrains destinations, and a structured router selects billing, diagnostics, or a decomposition workflow.

Runtime

An authenticated customer reports unexpected charges during an outage; the router recognizes two linked intents and creates bounded work for both.
The support interface shows the selected paths and allows a service analyst to correct routing with a recorded reason.

Failure scenario

When confidence is insufficient or a destination lacks required access, the request enters clarification rather than a random available worker.

Operational impact

Track task completion by route, correction rate, ambiguity rate, and routing overhead. Overall accuracy can hide failures in rare high-impact intents.

Architecture tradeoffs

Decision	Benefit	Cost
Model-based classification	Handles varied language and contextual intent.	Adds inference cost and requires ongoing calibration.
Rules before classification	Enforces stable eligibility and common cases cheaply.	Growing rule sets can become difficult to maintain.
Load-aware selection	Reduces queue delays when equivalent destinations exist.	Operational convenience must not override capability or policy.

Failure scenarios & recovery

Registry advertises a retired worker

Detect: Dispatch rejects the capability version or endpoint.

Recover: Refresh the registry and select a compatible eligible route.

Mixed intent is forced into one class

Detect: Repeated handoffs or unresolved subrequests appear in outcomes.

Recover: Use decomposition or ask a targeted clarification.

Common architecture mistakes

Treating router output as authorization

Enforce access independently at the destination.

Using unvalidated confidence thresholds

Calibrate on representative held-out requests.

Optimizing only classification accuracy

Include resolution quality, route cost, and escalation burden.

When to choose this pattern

Choose it when

Requests genuinely require different capabilities or workflows.
Eligibility depends on tenant, geography, or authenticated context.
Ambiguity can be handled through clarification or review.

Avoid it when

Every request uses the same tools and execution path.
Simple stable rules already achieve adequate routing quality.
No labeled outcomes exist to assess route correctness.

Architecture checklist

Version the capability registry and router policy.
Filter unauthorized routes before ranking.
Define an explicit mixed-intent route.
Calibrate thresholds on rare important classes.
Track route corrections with reason codes.
Test unavailable and unknown destinations.