11 — Tool Invocation & Function Calling

Concept overview

Tool invocation connects a language model to application capabilities through structured operation requests. The model proposes a function name and arguments; the application resolves, validates, authorizes, and executes that request. Producing syntactically valid arguments does not prove that the operation is permitted, sensible, or safe to retry.

Treat the tool layer as a business API boundary. Narrow tools with clear preconditions and typed results are easier to evaluate than unrestricted shell, SQL, or HTTP access. Distinguish read operations from commands, and represent pending, failed, and uncertain outcomes explicitly so the agent cannot interpret an empty response as successful execution.

Why it matters in production

Proof of concept

A prototype exposes a few read-only functions with hand-checked arguments. Evaluate tool selection and argument correctness separately from final-answer quality.

Production

Production introduces schema validation, scoped credentials, operation deadlines, result normalization, and idempotency for commands. Business invariants remain in the tool service even when the model follows instructions reliably.

Enterprise scale

At scale, tool registries require version compatibility, deprecation policies, and per-capability telemetry. Large tool catalogs should be filtered to relevant authorized capabilities to reduce selection errors and prompt overhead.

Architecture components

Tool registry

Defines stable operation names, argument schemas, side-effect classifications, and supported versions.

Invocation validator

Rejects unknown fields, invalid types, unsafe ranges, and unresolved business identifiers.

Authorization gateway

Checks the user's delegated authority against the specific resource and requested action.

Execution adapter

Applies deadlines, credentials, retry rules, and operation identity to the actual business API.

Result normalizer

Returns typed success, error, pending, or uncertain outcomes with source identifiers.

The architecture, mapped

flowchart LR
 M["Model proposal"] --> S["Schema validator"]
 S --> A["Resource authorization"]
 A --> I["Idempotency and deadline"]
 I --> T["Business API"]
 T --> N["Outcome normalizer"]
 N --> M
 S --> R["Rejected proposal"]
 A --> R
 I --> L["Operation ledger"]

Runtime flow

1. 1

   Expose capabilities

   Select only tools relevant to the task and permitted for the authenticated requester.

2. 2

   Receive proposal

   Parse the model's structured function request without evaluating arbitrary generated code.

3. 3

   Validate arguments

   Resolve identifiers and enforce schema, amount, date, and business-precondition checks.

4. 4

   Authorize action

   Recheck resource-specific permissions and require approval where the workflow demands it.

5. 5

   Execute bounded call

   Invoke the adapter with deadline and operation identity, recording the actual outcome.

6. 6

   Return observation

   Supply normalized evidence to the agent and reconcile unknown command outcomes before retrying.

Handling a damaged-item replacement

Retail · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

A retailer's support assistant helps agents replace damaged products when an order is eligible under policy.

The problem

A model can identify the right intent but propose the wrong order line, quantity, or delivery address.

Agent requirement

The assistant must retrieve verified order facts and create a replacement only within the customer's entitlement.

Architecture

Separate lookup and replacement tools expose narrow schemas; the command adapter checks eligibility and uses a stable replacement operation key.

Runtime

• The assistant retrieves the order, identifies the damaged line, and proposes a replacement quantity with source references.
• The gateway validates entitlement and address confirmation, then the order system returns the replacement identifier as authoritative evidence.

Failure scenario

If the creation request times out, the adapter queries the operation ledger before permitting another command.

Operational impact

Monitor wrong-tool selection, argument rejection, unauthorized attempts, and duplicate-command prevention alongside customer resolution time.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Executing arbitrary function names from model text

Resolve only registered capability identifiers.

Using one privileged credential for all callers

Apply delegated or resource-scoped authorization.

Retrying every command after timeout

Reconcile outcomes and use idempotency before retries.

When to choose this pattern

Choose it when

• Agents need current facts or controlled business operations.
• Capabilities can be expressed as bounded typed contracts.
• Application code can independently enforce business rules.

Avoid it when

• Static generation needs no external capabilities.
• Only unrestricted privileged execution is available.
• A critical command cannot expose verifiable completion semantics.

Architecture checklist

• Classify every tool as read or command.
• Validate arguments and results independently.
• Check authorization for each resource.
• Assign stable keys to retryable commands.
• Document uncertain-outcome recovery.
• Evaluate tool selection and argument accuracy separately.

Related concepts

• 9 — ReAct Framework (Reasoning + Acting)
• 22 — Role-Based Access Control (RBAC)
• 23 — Attribute-Based Access Control (ABAC)
• 41 — Idempotent Tool Execution
• 49 — Timeout Policies

Primary references

• JSON Schema: understanding JSON Schema
• Amazon EC2: ensuring idempotency