What is traceability?
Traceability is the ability to follow an output back through every input and every step that produced it. A trace records one request, where traceability spans the whole chain, including the document version read and the model version that read it. It is what makes an output explicable to someone absent when it was produced.
Also called: Lineage
Why the chain fails at its least governed link
Model risk functions and clinical governance committees ask versions of one question, which is whether a given output can be reconstructed from the inputs behind it. Answering requires every link to hold, and the chain fails at whichever link was treated as infrastructure rather than as evidence. The usual weak link is the source document, updated in place by a content owner never told that a past answer depends on the version they replaced. Traceability is an engineering commitment made before the first output is issued.
- One missing link makes the whole chain unusable as evidence.
- Source versioning is the link most often absent, and the hardest to add later.
- Traceability converts a model output into a defensible input to a decision.
- The property is tested by reconstruction rather than by documentation.
How traceability works
- Every source is versioned, with each superseded version retained and addressable.
- Each request records the exact source versions and the model version in use.
- Prompt and configuration revisions are identified and stored against the request.
- The output is stored with references to each input rather than a copy of it.
- Reconstruction is exercised on sampled outputs at a stated interval.
- Retention across every link is set by the longest applicable challenge window.
How traceability is evaluated
Traceability is proved by reconstructing outputs that somebody else chose. Examine the following on outputs already issued to users:
- proportion of outputs reconstructible to their exact inputs
- availability of each source at the version originally read
- presence of model and prompt version identifiers on every request
- elapsed time to reconstruct one output on demand
- consistency of retention across the weakest link in the chain
Limitations, risks and common failure modes
- Content owners edit source documents in place, so the version read has gone.
- A model version is replaced with no record of which outputs it produced.
- Prompt revisions are deployed from a configuration store keeping no history.
- The chain holds inside one system and breaks at an integration boundary.
- Retention is set per system, so the shortest setting governs the whole chain.