01 — Multi-Agent Orchestration
Coordinate specialist agents through explicit ownership, shared contracts, and bounded execution so a business task produces one accountable result.
Top 100 AI Agent Architecture Design Concepts
An enterprise architecture reference for designing, deploying, securing, scaling and operating AI agents.
Read 100 full lessons, 12 advanced extensions, and 28 industry blueprints. Each connects architecture components, runtime flows, Mermaid diagrams, failure handling, tradeoffs, and review checklists.
The architecture lifecycle
- Define the task and decision boundary.
- Connect tools and evidence.
- Secure identity, data, and actions.
- Operate with observability and recovery.
- Improve quality, cost, and capacity.
Five connected domains
Agentic Frameworks & Coordination
From a single decision to a coordinated team. Design how agents plan, delegate, remember, and act.
Enterprise-Grade Security & Governance
Build permission boundaries, protect enterprise knowledge, and make every action accountable.
Resilience, Reliability & Failure Handling
Make failure a designed part of the system. Recover work safely and keep useful service available.
Advanced RAG & Data Architecture
Connect agents to relevant, current, authorized knowledge. Design retrieval you can evaluate.
Model Optimization, Deployment & Infrastructure
Balance quality, latency, capacity, and cost. Turn model capabilities into dependable services.
Learning paths
Find your learning pathIndustry architecture library
Explore all 28 industry blueprintsAll 100 concepts
02 — Directed Acyclic Graphs (DAGs)
Represent bounded agent work as explicit dependencies, enabling reproducible scheduling, parallel branches, and precise restart boundaries.
03 — Agentic State Machines
Make agent progress an explicit set of states, guarded transitions, and durable events so every action has a permitted execution context.
04 — Hierarchical Orchestration
Delegate through supervisory layers that preserve domain ownership, limit context, and return evidence-rich results to a single accountable decision maker.
05 — Dynamic Routing
Select the appropriate agent or workflow from request intent, capability, authorization, and uncertainty while keeping routing decisions observable and reversible.
06 — Asynchronous Agent Execution
Move long-running agent work into durable jobs with explicit acceptance, progress, cancellation, and completion semantics.
07 — Agent Inter-communication (A2A)
Interoperate across independently operated agents through explicit capability discovery, task lifecycle contracts, authenticated messages, and verifiable artifacts.
08 — Plan-and-Execute Pattern
Separate deliberate task planning from controlled execution, validating dependencies and revising only the parts invalidated by new evidence.
09 — ReAct Framework (Reasoning + Acting)
Alternate bounded action proposals with tool observations, allowing an agent to adapt its next step while application code controls authority and termination.
10 — Reflection Loops
Use explicit critique and revision cycles to improve a draft against measurable criteria, while avoiding unsupported confidence from repeated self-assessment.
11 — Tool Invocation & Function Calling
Convert model-proposed operations into validated, authorized tool calls with stable schemas and explicit treatment of side effects.
12 — State Persistence
Persist the minimum authoritative execution state needed to recover an agent run, maintain concurrency safety, and explain completed work.
13 — Sub-Goal Generation
Decompose broad objectives into bounded sub-goals with explicit dependencies, measurable completion criteria, and a controlled expansion budget.
14 — Token-Aware Routing
Route work according to estimated context and output demand while preserving capability, evidence quality, and a hard task-level token budget.
15 — Event-Driven Agent Triggers
Start bounded agent work from meaningful business events using durable publication, deduplicated consumption, and explicit event-to-task ownership.
16 — Task Interruption & Resumption
Pause a durable execution at a defined boundary and resume it with validated input, current authority, and reconciled external effects.
17 — Agent Lifespan Management
Control how agent instances are created, leased, reused, drained, and retired so long-running work does not accumulate abandoned resources or authority.
18 — Semantic Condition Checking
Evaluate meaning-dependent workflow conditions through evidence-backed classification while keeping hard business predicates deterministic and reviewable.
19 — Context Condensation
Compress accumulated context into a versioned working summary that preserves commitments, evidence pointers, unresolved questions, and recovery-critical facts.
20 — Deterministic Output Stripping
Extract and validate an explicit public output contract so model responses cannot leak internal fields, malformed wrappers, or unintended executable content.
21 — Authorization-Aware RAG
Apply the requester’s current data entitlements throughout retrieval, reranking, generation, caching, and citation delivery.
22 — Role-Based Access Control (RBAC)
Assign stable permissions to organizational roles and enforce those permissions at every agent tool and data boundary.
23 — Attribute-Based Access Control (ABAC)
Evaluate access from trusted subject, resource, action, and environment attributes when static roles cannot express the required business boundary.
24 — Prompt Injection Mitigation
Constrain untrusted instructions at data boundaries and prevent retrieved content from acquiring tool authority, secrets, or control over the agent workflow.
25 — Data PII Masking
Detect and transform personal data before model exposure while preserving task utility, protecting reidentification mappings, and measuring missed entities.
26 — VPC Peering / Private Links
Connect agent workloads to private services through deliberately scoped network paths while retaining application authentication, encryption, and egress control.
27 — Responsible AI Governance (ISO/IEC 42001)
Translate responsible-AI commitments into an operating management system with named owners, risk decisions, evidence, and continuous improvement.
28 — Deterministic Data Grounding
Bind generated claims to verified records and use deterministic computation for values that must be exact, current, and reproducible.
29 — Output Jailbreaking Guardrails
Apply an explicit output policy before publication and prevent unsafe model content from becoming executable instructions or unauthorized business actions.
30 — Non-Repudiation Logs
Create attributable, tamper-evident records of agent decisions and actions while recognizing that evidence strength depends on identity, keys, and independent custody.
31 — LLM Evaluation Frameworks (TruLens/Ragas)
Evaluate agent and RAG behavior with versioned datasets, component metrics, calibrated judges, and release criteria tied to actual task outcomes.
32 — Model Bias Auditing
Measure whether model behavior and downstream decisions differ across relevant groups, investigate causes, and validate mitigations within the actual application context.
33 — Token Rate Limiting
Enforce token-based admission and consumption budgets so individual requests and tenants cannot exhaust shared inference capacity or spend.
34 — Tenant Isolation
Prevent one tenant’s identity, data, state, tools, or resource consumption from crossing into another tenant’s execution boundary.
35 — Adversarial Prompt Testing
Continuously test adversarial inputs against the application’s actual trust boundaries, measuring unauthorized effects and preserving reproducible regression cases.
36 — Watermarking AI Outputs
Attach or detect provenance signals in generated content while treating watermark evidence as limited, attack-sensitive, and distinct from proof of truth.
37 — Zero-Data Retention (ZDR) Contracts
Translate retention commitments into enforceable endpoint selection, feature restrictions, logging controls, and evidence that the deployed data path matches the agreement.
38 — Data Lineage Tracking
Track how source data becomes chunks, embeddings, prompts, model outputs, and decisions so changes and incidents can be traced to their affected artifacts.
39 — Content Moderation API
Integrate content classification as a versioned policy service with explicit thresholds, contextual handling, latency budgets, and review paths.
40 — Explainable AI (XAI)
Provide audience-appropriate explanations grounded in actual evidence, rules, and execution records without presenting generated rationales as faithful internal reasoning.
41 — Idempotent Tool Execution
Make repeated attempts at one business operation converge on one recorded effect, even when agents, queues, or networks deliver the request more than once.
42 — Circuit Breakers (Resilience4j)
Stop sending ordinary traffic to an unhealthy dependency, probe recovery deliberately, and protect the rest of the agent workflow from repeated failure.
43 — Self-Correction Loops
Repair specific validated defects in an agent’s work through bounded feedback, preserving accepted facts and preventing repeated unsafe actions.
44 — Human-in-the-Loop (HITL)
Place accountable human decisions at explicit workflow boundaries, presenting sufficient evidence and preserving approval scope through execution.
45 — Graceful Degradation
Preserve the most valuable safe subset of service during dependency failure, making reduced capability and evidence freshness explicit to users.
46 — Max Iterations Fallback
Terminate agent loops predictably when iteration, time, cost, or progress limits are reached, returning an honest partial outcome and a controlled next step.
47 — Fallback Prompt Strategies
Use evaluated alternate prompt strategies for specific recoverable failure classes while preserving policy, evidence requirements, and response contracts.
48 — Dead-Letter Queues (DLQ) for Agents
Quarantine repeatedly failing agent jobs with enough context for diagnosis and controlled replay, keeping poison work from blocking healthy processing.
49 — Timeout Policies
Allocate end-to-end deadlines across queueing, inference, tools, retries, and finalization so work stops predictably and uncertain side effects are reconciled.
50 — Exponential Backoff with Jitter
Retry transient failures with bounded exponential delays and randomized timing, preserving operation identity and respecting the remaining task deadline.
51 — State Rollbacks
Restore a coherent workflow checkpoint while explicitly compensating business effects that a database rollback cannot undo.
52 — Model Redundancy / Multi-Provider Failover
Continue eligible model work through independent providers without silently changing capability, privacy, or output contracts.
53 — Health Check Endpoints for Agents
Separate process survival, traffic readiness, and end-to-end agent capability so health automation makes the correct recovery decision.
54 — Memory Leak Mitigation in Long-Running Agents
Bound retained state and resource lifetimes so long-running agents remain stable as conversations, tools, and tenants accumulate.
55 — Distributed Tracing (OpenTelemetry)
Connect agent decisions, retrieval, model calls, and tool effects into one causal execution record using interoperable trace context.
56 — Load Balancing LLM Requests
Distribute inference work using token demand, queue pressure, capability, and cache locality instead of treating all requests as equal.
57 — Crash-Safe Memory
Persist agent progress and effect receipts so a replacement worker can resume safely after an abrupt process or machine failure.
58 — Partial Success Management
Represent completed, failed, and unknown sub-results explicitly so a workflow can deliver useful work without misrepresenting completion.
59 — Cascade Failure Prevention
Contain overload and dependency failures so one slow tool or model does not consume the capacity of an entire agent platform.
60 — Chaos Engineering for AI
Test explicit resilience hypotheses by injecting controlled infrastructure and AI-specific faults, then measuring whether workflow invariants survive.
61 — Semantic Caching (Redis/GPTCache)
Reuse validated answers for sufficiently similar requests while preserving tenant, policy, freshness, and task boundaries.
62 — Hybrid Search (Dense + Sparse)
Combine lexical precision with semantic recall, then fuse candidate rankings under one authorization and evaluation contract.
63 — Metadata Pre-Filtering
Restrict the searchable evidence set using trusted metadata before semantic ranking, while validating the store’s actual filtering behavior.
64 — Parent-Child Chunking
Retrieve precise child passages, then expand them into bounded parent context that preserves meaning, provenance, and access boundaries.
65 — Re-ranking (Cohere/Cross-Encoders)
Use a stronger query-document relevance model on a bounded shortlist before expensive generation consumes the evidence.
66 — Query Deconstruction / Expansion
Turn complex or underspecified questions into bounded retrieval plans while preserving the original intent and evidence requirements.
67 — Graph RAG / Knowledge Graphs
Combine entity relationships with source passages to answer connection-oriented questions that isolated similarity matches cannot resolve.
68 — Time-Weighted Vector Retrieval
Blend semantic relevance with task-appropriate freshness while distinguishing publication time, event time, and validity.
69 — Vector Database Scaling (Pinecone/Milvus)
Scale vector retrieval by separating storage, indexing, query capacity, tenancy, and ingestion while measuring recall under real constraints.
70 — Dynamic Prompt Assembly
Build each model request from versioned policy, authorized evidence, task state, and a deliberate token allocation.
71 — Context Window Optimization
Allocate finite model context to the evidence and state that matter, preserving constraints and enough capacity for a complete response.
72 — Self-RAG / Self-Reflection
Use bounded evidence critique to decide when retrieval or revision is necessary, while avoiding self-confidence as a substitute for verification.
73 — Cross-Lingual Retrieval
Retrieve evidence across languages using evaluated multilingual representations or translation paths while preserving source meaning and access scope.
74 — Hypothetical Document Embeddings (HyDE)
Generate a hypothetical relevant passage to improve retrieval alignment, then answer only from real retrieved evidence.
75 — Chunk Overlap Tuning
Tune shared text between neighboring chunks to preserve boundary evidence without filling retrieval results with duplicates.
76 — Multi-Aspect Indexing
Index different views of the same source so queries can match its content, purpose, entities, and structure without losing source identity.
77 — Real-time Vector Ingestion
Keep retrieval synchronized with changing source systems using versioned events, idempotent processing, and measurable visibility lag.
78 — Lost in the Middle Mitigation
Test and reduce position-sensitive evidence loss in long prompts through selection, ordering, local extraction, and explicit coverage checks.
79 — Embedding Model Fine-Tuning
Adapt the retrieval representation to domain relevance using trustworthy examples, hard negatives, and a controlled re-indexing migration.
80 — Token Pruning
Remove low-value prompt content under a measured quality constraint while preserving negation, identifiers, evidence, and instructions.
81 — Dynamic Model Routing
Select an eligible model for each task using measured capability, quality, latency, and cost while keeping policy constraints mandatory.
82 — Small Language Models (SLMs) Deployment
Deploy compact models for bounded tasks where local control, latency, or operating cost matters and measured capability is sufficient.
83 — Speculative Decoding
Use a cheap proposal mechanism and target-model verification to reduce decoding latency only where acceptance and workload conditions justify the overhead.
84 — Model Quantization (AWQ/GGUF)
Reduce model representation cost using hardware-compatible quantization while validating task quality, memory headroom, and actual serving speed.
85 — LLM Cold Start Optimization
Shorten the path from a scheduled inference replica to usable capacity by measuring and optimizing each initialization stage.
86 — Streaming LLM Responses
Deliver incremental model output with explicit completion, cancellation, validation, and reconnect semantics rather than treating chunks as a finished answer.
87 — Batch Prompting
Amortize work across independent items while preserving item identity, isolation, validation, and retry boundaries.
88 — vLLM / TensorRT-LLM Engines
Treat the inference engine as a scheduled memory-and-compute system whose configuration must match models, hardware, and service objectives.
89 — Serverless LLM Architecture
Separate request admission from elastic inference capacity so serverless execution can scale safely despite model startup and duration constraints.
90 — LoRA / QLoRA Fine-Tuning
Adapt a base model through low-rank trainable updates, optionally over a quantized frozen base, with explicit compatibility and evaluation controls.
91 — GPU Cluster Orchestration (Kubernetes/KServe)
Coordinate accelerator placement, model readiness, autoscaling, and recovery as one serving system rather than treating GPUs as ordinary interchangeable CPU slots.
92 — Pre-Computing Embeddings
Compute reusable document vectors ahead of requests using content identity, version lineage, and an explicit freshness strategy.
93 — Model Versioning and A/B Testing
Version the complete inference behavior and compare releases through stable assignment, meaningful outcome measures, and reversible promotion.
94 — Token Cost Allocation Tracking
Attribute inference usage to the business work that caused it, including retries, caches, tools, and shared infrastructure.
95 — Distributed Memory Architectures
Distribute model weights, activations, and attention state across devices with an explicit communication and failure model.
96 — Semantic Similarity Analytics
Analyze semantic neighborhoods to find retrieval gaps, repeated intents, and content drift without treating vector distance as an objective truth score.
97 — Hardware-Accelerated Guardrails
Accelerate selected safety and policy checks while keeping their detection limits, ordering, and failure behavior explicit.
98 — Local Model Ingress/Egress
Control how requests enter a locally hosted model and how data, tools, telemetry, and updates leave its environment.
99 — Context Caching (Anthropic/OpenAI Feature)
Reuse provider-supported computation for stable prompt prefixes while preserving fresh generation and provider-specific eligibility, retention, and billing rules.
100 — Asynchronous Fine-Tuning Pipelines
Run data preparation, adaptation, evaluation, and model promotion as a durable offline workflow with explicit quality gates and lineage.