Explainable Credit Decisions: Generating Adverse-Action Notices From AI Models

The failure that defines this problem is the plausible-but-unfaithful reason. It is easy to produce reasons that sound like adverse-action reasons, and it is hard to produce reasons that actually reflect what drove the model’s decision, and a notice that gives plausible reasons that are not the real ones is not compliance, it is a defensible-looking answer that collapses when a regulator reconstructs the decision.

The constraint that owns the workflow

Adverse-action requirements are specific and unforgiving. A declined applicant is entitled to the specific principal reasons for the denial, which have to be accurate and reflect the actual basis of the decision, delivered within a defined timeframe. The reasons cannot be generic or misleading, and they have to be specific enough for the applicant to understand what drove the denial. The model may not base decisions on prohibited characteristics or proxies for them, and the adverse-action reasons have to be consistent with a decision that did not, so the explanation is bound up with the fair-lending obligation. And the decision and its reasons have to be reconstructable, so the lender can defend that the reasons given were the real ones.

The failure modes are specific to explaining a model’s decision. The unfaithful reason, where the notice states reasons that sound plausible but do not reflect what the model actually weighed, which fails the accuracy requirement even though it looks compliant. The generic reason, where the notice gives reasons too vague to be the specific principal reasons the applicant is owed. And the inconsistent-explanation failure, where the reasons given do not match the actual decision basis in a way that surfaces a fair-lending problem or an unexplainable decision when examined.

The position: faithfulness is the requirement, and plausibility is not enough

Here is the stance I would hold. An adverse-action notice has to state the reasons that actually drove the model’s decision, and a plausible-sounding reason that does not faithfully reflect the model’s actual reasoning does not satisfy the obligation, it violates it while appearing to comply. The requirement is faithfulness, the reasons given must be the real ones, and this is a higher bar than plausibility, because it is easy to generate reasons that look like adverse-action reasons and hard to generate reasons that are genuinely what the model weighed. A system that produces plausible but unfaithful reasons has built the lender a compliance failure that is invisible until a regulator reconstructs a decision and finds the stated reasons were not the actual ones.

The reasoning is about what the obligation is actually for and how it is tested. The adverse-action requirement exists so the applicant learns the real reasons they were declined, so they can understand and potentially address them, and so the decision can be checked for fairness. A notice that gives reasons which are not the real ones defeats all of this, misleading the applicant and misrepresenting the decision, and when a regulator examines the lender’s decisions, they can reconstruct what the model actually did and compare it to the reasons the notice gave, so an unfaithful explanation is discoverable and damning. This means the explanation method has to produce reasons that genuinely reflect the model’s decision for each specific applicant, which is a real technical requirement when the model is complex, because faithful explanation of a complex model’s individual decision is hard, and the easy explanation methods often produce plausible approximations rather than faithful accounts. So the engineering has to prioritise faithfulness, verifying that the reasons given actually reflect what the model weighed, because a plausible explanation that is not faithful is worse than useless, it is a liability that looks like compliance.

Engineering inside the constraint

The explanation method is chosen and built for faithfulness to the model’s actual per-decision reasoning. For each denial, the system produces the specific principal reasons that reflect the factors the model actually weighed most heavily for that particular applicant, traceable to the model’s actual decision rather than a generic account of what the model tends to weigh. This per-decision faithfulness is the requirement, because the applicant is owed the reasons for their decision, not a general description of the model, and the faithfulness of the explanation to the actual decision has to be a verified property rather than an assumed one.

The reasons are rendered into the specific, accurate, understandable form the notice requires, so the applicant receives principal reasons they can actually understand and act on rather than technical factors or vague generalities. The explanation and the decision are kept as a single traceable artifact, so the reasons given, the decision made, and the factors the model weighed are one reconstructable record rather than separate accounts that might not match. A reconstructable decision-and-explanation record is what lets the lender defend that the reasons given were the real ones when a regulator reconstructs the decision.

The explanation faithfulness is monitored, because an explanation method can be faithful on some decisions and less faithful on others, or can drift as the model changes, and the lender has to know its explanations remain faithful across the decisions it makes. The system is evaluated against an explicit standard measuring whether the generated reasons faithfully reflect the model’s actual decision basis, operated repeatably as the model changes, which is the work a rubric-based evaluation engine like PRISM is built to carry, because an explanation capability that drifts out of faithfulness produces notices that quietly stop being compliant. The consistency of the explanations with fair-lending compliance is checked, since the reasons should reflect a decision free of prohibited factors and proxies. Human oversight sits on the adverse decisions, where the reasons feed the applicant’s understanding and any dispute, and where a person can ensure the explanation and the decision hold together.

Where it almost broke

A lender deployed a complex credit model that improved risk prediction, and to meet the adverse-action requirement, it added an explanation layer that generated principal reasons for each denial. The reasons read well, they were specific and plausible and looked exactly like proper adverse-action reasons, and the lender considered the obligation met. The explanation method, though, produced reasons based on a simplified approximation of the model rather than the model’s actual per-decision reasoning, so the reasons were plausible accounts of why an applicant like this might be declined rather than faithful statements of what the model actually weighed for that applicant. When a regulator examined a sample of decisions, reconstructing what the model had actually done, the stated reasons did not match the actual decision basis for a number of applicants, and the lender was found to have given reasons that were not the real ones.

The reasons looked compliant and were not faithful, and the gap between plausible and faithful was the compliance failure. The fix was to replace the explanation method with one that produced reasons faithful to the model’s actual per-decision reasoning, traceable to what the model actually weighed for each applicant, and to monitor explanation faithfulness so it did not drift. The notices became genuinely faithful rather than plausibly approximate, and the compliance exposure closed. An adverse-action notice that gives plausible reasons which are not the model’s real reasons is a compliance failure that looks like compliance until a regulator reconstructs the decision.

Explainable credit decisions come down to a hard requirement that AI models make harder: for every denial, the lender has to generate an adverse-action notice whose reasons are accurate, specific, and faithful to what the model actually did. The complexity that makes modern credit models accurate is exactly what makes faithful explanation difficult, and a system that produces plausible-but-unfaithful reasons has built a compliance failure that looks like compliance until a regulator reconstructs a decision and finds the stated reasons were never the real ones. Faithfulness, verified and monitored, is what makes an adverse-action notice defensible.

A lender generating adverse-action notices from AI models should ask whether the reasons it gives genuinely reflect what the model weighed for each specific applicant, or whether they are plausible accounts that would not survive a regulator reconstructing the decision. An adverse-action notice whose reasons are not the model’s real reasons is not an outcome that survives scrutiny.