Model Risk Governance Beyond the Validation Team

What is model risk?

Model risk is the potential for adverse outcomes when a model is wrong, misused or drifts away from the conditions it was built for. Banking supervisors have long required controls for it, typically an inventory, development standards, independent validation, ongoing monitoring and governance by senior management. AI extends the same concerns: more models, faster change, more third-party components, and outputs that are harder to specify.

Why not leave it to the validation team?

A central team cannot review every model in an AI-heavy organization, and it should not be the only owner of quality. Developers decide how a model is built, business owners decide how it is used, and monitoring happens in production. Governance works when each line has clear duties: those who build and own the model document and test it, an independent function challenges it in proportion to risk, and internal audit checks that the whole arrangement runs as described.

What does a workable lifecycle look like?

• Inventory and tiering: every model is registered and rated for materiality, with the AI inventory as the source.
• Development standards: documented purpose, data, method, limitations and tests, kept with the model version.
• Independent challenge: depth scaled to tier, covering conceptual soundness, data, performance and sensitivity to change.
• Release approval: conditions and limits of use are stated, and data checks pass a release gate.
• Monitoring: performance, drift, overrides and complaints are tracked against thresholds with named owners.
• Change management: retraining, new data, new vendor versions and new uses trigger review.
• Retirement: models are decommissioned deliberately, with records retained.

What changes for generative and vendor models?

You may not see a vendor model's training data or internals, so the evidence shifts to what you can observe: evaluation on your own tasks, red-team testing, input and output controls, contractual commitments and a record of which version you used. Because outputs vary, validation relies on representative test sets, scoring rubrics and periodic sampling of live behavior, not a single accuracy figure. Prompts, retrieval sources and guardrails belong under version control like code. For systems that act on their own, see governing agents and RAG.

A hypothetical walk-through

A wealth manager adds a generative assistant that drafts client summaries. This example is hypothetical.

1. The assistant is tiered as moderate: advisers review every draft, but errors could mislead clients.
2. Developers document the prompt design, approved sources and known failure modes.
3. An independent reviewer builds a test set from past cases, checks for unsupported claims, and sets a threshold for release.
4. In production, a sample of drafts is reviewed monthly, and adviser edits are logged as a drift signal.
5. When the vendor announces a new model version, the change process reruns the test set before the switch.

How does this connect to wider frameworks?

Model risk practice maps neatly to the Measure and Manage functions of the NIST AI RMF and to the operational controls in an AI management system. Supervisory expectations vary by jurisdiction and sector, so confirm with your regulators which guidance applies to you.

What evidence should the governance forum see?

A model risk or AI governance council works better with a regular, compact pack than with ad hoc papers. Useful contents include the inventory by tier, validation findings and their age, models running outside agreed limits, open exceptions, incidents and near misses, and upcoming changes such as vendor model retirements. Trend matters more than any single number: a rising count of overdue findings tells the council more than the count alone. Record decisions and the reasons for them, since those minutes are evidence in themselves.

What are common pitfalls?

• Applying one heavy process to every model, so low-risk work is slowed and high-risk work is not examined more deeply.
• Treating vendor models as outside scope because the code is not visible.
• Validating once at launch and never again as data and behavior change.
• Leaving ownership unclear when a model is shared across business units.

How do you size the effort?

Match review depth to tier. A high-tier credit or clinical model may warrant a full independent validation with replicated results, while a low-tier internal summarizer may need a documented test set and an annual check. Publish the expectations for each tier so teams can plan for them, and review the tiering itself once a year.

Finally, invest in skills. Challenge is only as good as the reviewers, so train validators on generative techniques and give developers a clear template for what a good model document contains.

Where aerais fits

aerais helps organizations extend model risk practice to AI: tiering, standards, monitoring and evidence, built into delivery pipelines and then operated alongside the governance function. Model-risk governance is part of our AI governance practice, and the method and framework describe how the pieces are built and mapped.

This article is for general information only and is not legal advice.