The Evidence Did Not Survive.
A lending decision survived operationally and failed evidentially.
A retail bank introduced an AI-assisted decision service into its unsecured personal-loan process. The system ranked applications by predicted credit risk, recommended approve, decline or refer, and routed borderline cases to human credit analysts.
The AI did not initially make the final legal decision. The bank retained formal responsibility for the outcome.
The approval was careful. Risk had reviewed the model. Compliance had considered customer-treatment implications. Technology had tested the integration. Credit had agreed the decision thresholds. Operations retained a manual-review step. A three-month pilot had shown lower processing time, stable arrears and no material deterioration in approval quality.
The Credit Risk Committee approved full deployment.
Six months later, complaints began to cluster around one group of applicants. Senior management could see that the system was behaving consistently. What it could not explain quickly was a more difficult question:
Why had the bank originally concluded that this particular decision framework was safe and fair enough to approve?
The answer took eight working days to reconstruct.
Approval record
The committee paper, approval date and accountable executive were available. Deployment had been formally approved.
Decision records
The bank could retrieve application data, credit-risk scores, model recommendations, referrals, overrides and final customer outcomes.
Model documentation
Validation reports, supplier materials and the current model documentation remained available.
At first glance, the evidence base looked comprehensive.
It was not.
The assumptions
The pilot assumed a broadly stable applicant mix, manageable referral volumes, and enough analyst capacity for meaningful human review. Those assumptions were discussed but not attached to the approval.
The rejected alternative
A slower rollout limited initially to existing customers had been considered and rejected. The final minutes recorded the decision, not why that alternative lost.
The challenge
A senior credit analyst had questioned whether the exception threshold would remain workable if referral volumes rose. The threshold survived. The challenge did not.
The approval context
The supplier had made incremental model updates. The bank could identify the current version, but not quickly prove whether the original approval assumptions still held after those changes.
The bank could reconstruct how a customer was scored. It could not reconstruct the judgement that made the overall lending framework acceptable.
The control still existed. The condition underneath it had changed.
The board asked one question:
If we approved this because borderline customers would receive meaningful human judgement, can we demonstrate that the human judgement still exists in practice?
A human reviewer was still formally present. The workflow had not removed the human. The policy had not changed. The approval had not expired.
But application volumes had risen, the customer mix had shifted and average review time per referred case had fallen materially.
Nothing in the original approval defined a minimum review condition. Nothing required the decision to be reopened if analyst capacity deteriorated.
The bank therefore had evidence that the control still existed. It had much weaker evidence that the control was still doing what the approval had assumed it would do.
| DI component | Financial-services finding |
|---|---|
| Authority | Formally clear. Credit Risk owned the lending framework, and the accountable executive remained identifiable. |
| Evidence | Partial. Approval papers and validation survived, but the decision was not linked to material business assumptions. |
| Boundaries | Formally stable, practically shifting. Human review remained in the workflow, but workload reduced how much meaningful judgement occurred. |
| Traceability | Strong for individual customer transactions, weak for reconstructing why the overall decision framework had been judged acceptable. |
| Challenge | A meaningful pre-launch challenge occurred, but the challenge itself left little durable evidential trace. |
| Reopening | No explicit trigger required reconsideration when applicant mix, model version, referral volume or reviewer capacity changed. |
The transaction trail was strong. The decision trail was not.
What it could not reconstruct quickly was the wider decision trail:
- Why this model?
- Why this threshold?
- Why this level of human review?
- Which assumptions made the arrangement acceptable?
- Which alternative was rejected?
- Which challenge was raised?
- What change should have caused the approval to be reconsidered?
An audit trail helps reconstruct activity. A decision trail must also help reconstruct judgement, authority and accountability.
Forensic reconstruction
Rebuild the decision only when challenged. Lowest immediate effort. Highest delay and dependence on institutional memory.
Decision Evidence Pack
For material AI-enabled credit decisions, preserve a compact record of facts, assumptions, challenge, authority, rationale and reopening conditions at approval.
Decision Receipt
Link minimum decision evidence to the operating environment, including model context, policy version, accountable owner and reopening triggers.
The bank moved from Option A to a hybrid of B and C. The objective was not to document everything. It was to stop losing the material evidence that made the decision defensible.
Approval stopped being treated as permanent.
Decision assumptions
The approval now records assumptions such as expected referral volumes, reviewer capacity, applicant population characteristics, acceptable override patterns and model-performance ranges.
Human-AI boundary
The bank explicitly documents what the model may recommend and what humans must review, challenge, override, or approve.
Reopening triggers
The decision must return for reconsideration when material conditions change, including:
- model performance outside approved tolerance;
- a material model-version change;
- a material shift in applicant mix;
- referral volumes beyond reviewer capacity;
- significant changes in override patterns;
- complaint patterns indicating a new decision effect;
- changes in regulation or internal credit policy.
- If our AI-assisted credit decision were challenged six months from now, could we explain why the decision framework was reasonable when approved?
- Which assumptions are important enough that the approval depends on them remaining true?
- Can we identify the exact model, policy, threshold, and human-review conditions that applied at the time?
- Which challenges or dissenting views should leave a durable evidential trace?
- Where does human review formally exist but no longer provide meaningful decision capacity?
- Which changes automatically require the lending decision to be reopened?
- Are we monitoring only model performance, or also the conditions that made the original decision defensible?
That's The Decision Layer Compass™ for this month.
Reconstructing one AI-assisted lending decision should take minutes.
Discovering six months later that you can reproduce the score, the recommendation, and the customer outcome, but still need eight working days to explain why the framework was considered acceptable, takes rather longer.
The expensive part is reconstructing the assumptions that mattered, the alternative that was rejected, the challenge that was raised, the human-review condition that quietly weakened, and the changes that should have reopened the approval.
This case exposes that distinction.
𝐀 𝐬𝐲𝐬𝐭𝐞𝐦 𝐜𝐚𝐧 𝐫𝐞𝐦𝐚𝐢𝐧 𝐨𝐩𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐚𝐥𝐥𝐲 𝐜𝐨𝐧𝐭𝐫𝐨𝐥𝐥𝐞𝐝 𝐰𝐡𝐢𝐥𝐞 𝐭𝐡𝐞 𝐞𝐯𝐢𝐝𝐞𝐧𝐜𝐞 𝐭𝐡𝐚𝐭 𝐨𝐧𝐜𝐞 𝐦𝐚𝐝𝐞 𝐢𝐭 𝐝𝐞𝐟𝐞𝐧𝐬𝐢𝐛𝐥𝐞 𝐪𝐮𝐢𝐞𝐭𝐥𝐲 𝐞𝐱𝐩𝐢𝐫𝐞𝐬.
Forward this to a CIO, CRO, CAE, Chief Credit Officer, General Counsel, or Audit Committee Chair who is confident they can reconstruct what the AI did, but less certain they can reconstruct why the organisation decided it was acceptable to let it do it.
Next in the cycle
Regulatory Radar, 24 November. What Must Survive the Decision? Replies to this edition are read by the author.