Model Risk Management in the Age of Generative AI

Back to Blog·Risk Management

Model Risk Management in the Age of Generative AI

Dr. Sarah Chen
Head of AI Governance
8 min read

Traditional model risk management frameworks were designed for statistical models. Generative AI introduces fundamentally different risks — hallucination, prompt injection, and emergent behaviours — that require updated governance approaches.

Why Traditional MRM Falls Short

Traditional model risk management frameworks — such as SR 11-7 — were designed for statistical and machine learning models with well-defined inputs, outputs, and performance metrics. Generative AI models operate differently: they produce open-ended outputs, are sensitive to prompt phrasing, and can exhibit emergent behaviours not present in training.

New Risk Categories for GenAI

Organisations must extend their model risk taxonomy to include hallucination risk (factually incorrect outputs), prompt injection attacks, data leakage through model outputs, copyright and IP risks from training data, and reputational risks from inappropriate content generation.

Updated Validation Approaches

GenAI validation requires red-teaming exercises, adversarial testing, output quality benchmarking, and human evaluation panels. Automated evaluation metrics alone are insufficient — human oversight remains essential for high-stakes use cases.

Tagged:Model Risk

Ready to Strengthen Your AI Governance?

Take our free AI Governance Assessment or speak with one of our experts.