Defensibility

Prism

Whether a decision can be explained and substantiated when it is challenged, which is the only standard that matters at the point of appeal.

What is defensibility?

Defensibility is whether a decision can be explained and substantiated at the moment it is challenged. It asks nothing about whether the decision felt reasonable at the time. The question is whether the reasoning and its evidence remain available and coherent. It is the standard that applies at an appeal and at any regulatory review that follows.

Why accuracy and defensibility are different properties

A decision can be correct and indefensible at the same time, which is the distinction most teams miss. Correctness is a property of the outcome, where defensibility is a property of the record left behind, and a process can be excellent at one while failing at the other. The organisations that discover this do so at the point of challenge, when the right answer exists and the evidence for it has gone. Everything that makes a decision defensible has to be captured while the decision is being taken.

  • Defensibility is determined by the record rather than by the quality of the judgement.
  • It can only be established in advance, since evidence cannot be recreated afterwards.
  • A defensible process reduces the cost of each challenge as well as its risk.
  • It is the property a board and a regulator both actually test.

How defensibility works

  • The criteria and their weightings are fixed and published before assessment begins.
  • Each score is linked to the passage of evidence that supports it.
  • Approvals and overrides are recorded with their authorship and a timestamp.
  • Inputs are preserved at the version in which they were read.
  • Outcome rates are tested across groups and the result is retained with the cycle.
  • The complete reconstruction is rehearsed before any challenge arrives.

How defensibility is evaluated

Defensibility is proved by rehearsal rather than by assertion. Examine the following on decisions already communicated:

  • time required to produce a full reconstruction on demand
  • proportion of scores with linked and retrievable evidence
  • availability of the rubric version applied to each cohort
  • appeal overturn rate and the stated reason for each overturn
  • completeness of group outcome testing for the cycle

Limitations, risks and common failure modes

  • Teams measure accuracy continuously and never once test reconstruction.
  • Evidence links survive inside the platform and are lost on export.
  • The published criteria and the applied criteria diverge over a long cycle.
  • Defensibility is treated as a documentation task and assigned after go live.

Related terms

Decision recordThe complete account of a decision, from its inputs to its named author, retained for as long as the decision can be questioned.Reasoning traceThe recorded path from evidence to score, which is what makes a decision reviewable afterwards.Append-only logA record that can be added to and never amended, which is what makes a history worth presenting.AI audit trailThe complete record of what a system did, on what evidence, and who authorised it, retained for the period the decision can be questioned.Completion recordThe record of who completed which module, on which version of the source, and when.Cut-off thresholdThe score at which a decision changes, and the justification for placing it where it was placed.

All 315 terms