27 — Responsible AI Governance (ISO/IEC 42001)

Concept overview

ISO/IEC 42001 concerns an organizational AI management system: how an organization establishes, operates, maintains, and improves its management of AI. It is not a runtime guardrail library or a claim that an individual model is harmless. Architecture contributes evidence and enforceable controls to that management system.

Treat governance as an operating loop spanning inventory, impact assessment, risk treatment, release decisions, monitoring, incidents, and change review. The exact implementation depends on organizational scope and obligations. The public ISO overview describes the standard's purpose; detailed conformity work requires the applicable standard and qualified organizational review.

Why it matters in production

Proof of concept

A prototype needs an owner, intended-use statement, and basic risk record. These make later decisions traceable without creating a ceremonial approval process for every experiment.

Production

Production defines release criteria, accountable risk acceptance, evaluation evidence, supplier dependencies, and incident procedures. Controls should map to identified risks and measurable operating signals.

Enterprise scale

At scale, maintain a current AI inventory and reuse evidence across systems while preserving system-specific context. Management review must respond to incidents, changing use, and control failures rather than merely collecting documents.

Architecture components

AI system inventory

Records purpose, owners, users, data, models, suppliers, and lifecycle status.

Risk and impact register

Connects foreseeable harms and affected parties to treatment decisions and accountable owners.

Control implementation

Provides technical and organizational measures with testable operating evidence.

Release governance

Reviews material changes against approved criteria and documents risk acceptance.

Monitoring and review

Feeds incidents, evaluations, and audit findings into corrective action.

The architecture, mapped

flowchart LR
 I["AI inventory and intended use"] --> A["Impact and risk assessment"]
 A --> C["Owned controls and evidence"]
 C --> R["Release decision"]
 R --> O["Operational monitoring"]
 O --> M["Management review"]
 M --> A
 O --> X["Incident and corrective action"]
 X --> C

Runtime flow

1. 1

   Define scope

   Identify the AI system, intended users, operating context, and responsible organizational roles.

2. 2

   Assess impacts

   Examine foreseeable consequences and document assumptions, affected groups, and uncertainty.

3. 3

   Select treatment

   Assign controls, acceptance criteria, owners, and evidence requirements to identified risks.

4. 4

   Review release

   Evaluate test results and unresolved risks before the accountable release decision.

5. 5

   Monitor operation

   Collect relevant performance, safety, access, and incident signals with escalation thresholds.

6. 6

   Improve controls

   Investigate failures, verify corrective actions, and update the system's risk and use assumptions.

Governing an underwriting support assistant

Insurance · Illustrative enterprise scenario, not a reported customer outcome.

Business situation

An insurer introduces an assistant that summarizes applications for underwriters across several product lines.

The problem

Teams evaluate model quality independently but lack a shared record of intended use, supplier changes, and unresolved risks.

Agent requirement

The assistant must operate within a documented support role with accountable decisions about data, bias, reliability, and human oversight.

Architecture

A system inventory links model and prompt versions to risk assessments, evaluation suites, release records, and operational monitoring.

Runtime

• A proposed model change triggers a scoped review of affected capabilities and the evidence needed for release.
• After deployment, reviewer corrections and incidents feed the risk register and may require new tests or narrower operating limits.

Failure scenario

If monitoring shows a recurring unsupported exclusion summary, the owner restricts that feature and verifies remediation before restoring it.

Operational impact

Measure overdue control actions, evidence freshness, unresolved risk ownership, and detection-to-remediation time. This example illustrates governance architecture and does not claim certification.

Architecture tradeoffs

Failure scenarios & recovery

Common architecture mistakes

Equating a standard with a model safety guarantee

Distinguish management-system scope from system performance.

Collecting evidence without decision owners

Assign accountable acceptance and corrective-action roles.

Reviewing only initial deployment

Trigger review for material changes and incidents.

When to choose this pattern

Choose it when

• AI systems need accountable organizational risk management.
• Multiple teams share models, data, and supplier dependencies.
• Release and operating decisions require traceable evidence.

Avoid it when

• The goal is a single technical safety mechanism.
• A certification claim lacks the required assessment process.
• Documentation is being substituted for operating controls.

Architecture checklist

• Define system scope and intended use.
• Name risk and control owners.
• Map controls to identified impacts.
• Version release evidence and decisions.
• Establish incident and change-review triggers.
• Verify corrective actions through operating evidence.

Related concepts

• 30 — Non-Repudiation Logs
• 31 — LLM Evaluation Frameworks (TruLens/Ragas)
• 32 — Model Bias Auditing
• 35 — Adversarial Prompt Testing
• 38 — Data Lineage Tracking

Primary references

• ISO/IEC 42001:2023 overview
• NIST AI Risk Management Framework