Model Risk Management for Generative AI: Extending SR 11-7 to LLMs

The stakes are not academic. A bank that deploys an LLM into a customer-facing or decision-influencing role has put a model into production, and the supervisory expectation is unambiguous: it is subject to model-risk management like any other. The framework does not get a generative exemption. The question is how to apply a framework that assumes properties this class of model does not have.

The constraint that owns the workflow

SR 11-7 sets expectations that a bank cannot opt out of because the model is new in kind. A model must have a documented purpose and sound conceptual design. It must be independently validated before use, with that validation testing whether the model works as intended and identifying where it fails. Its performance must be monitored on an ongoing basis. And a named owner must be accountable for it across its life. These expectations were written for statistical models, and every one of them still applies to a generative system. The framework is not the problem.

The problem is that the standard validation techniques underneath the framework quietly assume determinism and a measurable target. Backtesting assumes a repeatable output you can compare to a realised outcome. Benchmarking assumes a defined performance metric. Sensitivity analysis assumes that small input changes produce traceable output changes. Point a validator armed with these techniques at an LLM and the techniques do not fail loudly. They produce numbers that look like validation while missing the risks that actually matter, which is the more dangerous failure because it manufactures false comfort.

Where classic validation goes blind

Three properties of generative systems defeat the inherited toolkit, and naming them precisely is the first step to validating around them.

Non-determinism is the obvious one. A model that can produce different outputs for identical inputs cannot be validated by checking a single output against an expected answer. A one-shot test tells you what the model did once, and infers nothing reliable about what it will do on the next call. Validation has to shift from testing outputs to testing the distribution of outputs: operating the same input many times and evaluating the range, the worst case, and the rate of unacceptable responses, because the risk lives in the tail, not the average.

The unbounded output space is the deeper problem. A credit model outputs a number in a known range. A generative model can output essentially anything, including responses no test suite anticipated. You cannot enumerate the failure modes in advance, which means validation cannot be a fixed checklist of expected behaviors. It has to include adversarial probing, out-of-distribution testing, and evaluation against the categories of harm the model could produce in its specific deployment, on the understanding that the space can never be fully covered.

Confident fabrication is the failure mode with the sharpest regulatory edge. A statistical model that is uncertain typically signals it through a wide confidence interval. A generative model that has left its competence produces a fluent, authoritative answer that is wrong, with no native uncertainty signal a validator can read. In a banking context that means the model is at its most dangerous precisely when it looks most reliable, and validation has to specifically test the boundary conditions where fabrication occurs rather than the comfortable middle where every model performs.

Extending the framework without stalling it

The instinct when a model class defeats your validation techniques is to refuse it until you are certain, and that instinct will keep a bank out of production indefinitely while competitors learn in the field. The workable path treats validation as an extension of SR 11-7 principles onto new techniques, keeping the principles fixed and rebuilding the methods underneath them.

Conceptual soundness still applies, reinterpreted for the deployment: the validation asks whether a generative model is the appropriate tool for this specific use, with its boundaries and fallbacks defined, before it asks how well the model performs. Ongoing monitoring still applies, extended to watch output distributions and drift in behavior rather than a single performance metric, because a generative model’s behavior can shift as its inputs and usage patterns change even when the model weights do not. Independent validation still applies, carried out through distributional and adversarial evaluation rather than backtesting against a point target.

The mechanism that makes this repeatable is a defined evaluation standard applied identically to every version of the model. Validating a generative system means scoring it against an explicit rubric that covers accuracy within its intended domain, behavior at the edges of its competence, refusal and fallback when it should not answer, and safety across the harm categories relevant to the deployment. Operating that evaluation as a gate the model clears before production, and repeating it automatically on every change, is what converts one-time comfort into continuous assurance. This is the work a rubric-based evaluation engine like PRISM is built to carry, so a prompt change, a model upgrade, or a retrieval-source update is re-validated against the full standard rather than trusted because a single sampled output looked reasonable.

Governance has to sit in the architecture here, not in a document filed at approval. Every generative decision that informs a customer or business outcome needs a logged record of the input, the output, and the context that produced it, so behavior can be reconstructed when a validator, an auditor, or an examiner asks. Human oversight placed by consequence keeps the model’s confident fabrication from reaching a customer unchecked in the uses where being wrong carries real cost.

Where it almost broke

A bank stood up a validation process for a generative assistant and operated it exactly as its model-risk framework prescribed. The validators tested a comprehensive set of expected queries, confirmed the answers were accurate, documented the results, and approved the model. The validation was thorough by the standards of the framework they had. It also tested each query once, and every query was one a competent model would handle well.

In production, the model encountered inputs the validation set never contained: ambiguous phrasings, questions just outside its knowledge, adversarial attempts to pull it into unauthorised territory. On those, its behavior was inconsistent across repeated calls, and on a subset it produced confident answers that were wrong. Nothing in the validation had tested repeatability or probed the boundary, so the risk was invisible until customers found it.

The fix was to rebuild the validation methods while keeping the framework. Each test input was operated many times to characterise the output distribution rather than a single response. The evaluation set was expanded with adversarial and out-of-distribution cases designed to find the boundary rather than confirm the center. And monitoring was extended to track behavioral drift in production, not just uptime. A validation process that tests a generative model the way it tested regression models will approve a system it never actually examined.

Model risk management for generative AI does not call for a new rulebook. It calls for the existing one to be applied with validation methods rebuilt for models that behave in ways deterministic techniques cannot see. The principles of SR 11-7, conceptual soundness, independent validation, ongoing monitoring, and clear ownership, all survive the transition intact. What changes is the work behind each principle, and a bank that keeps the principles while modernising the methods can bring generative systems to production without either reckless speed or indefinite paralysis.

The test of a generative validation process is not how many expected questions it answered correctly. It is whether it examined how the model behaves when it is uncertain, adversarially pushed, and asked something it was never trained for, because that is where the risk lives and where an examiner will look. A validation that only confirms the model works when it was always going to work has documented confidence without earning it, and confidence a bank has not earned is not an outcome that survives scrutiny.