The AI validation cycle

A board approves an AI system on a date. The data it was fitted to has a date, the process it was fitted to has a date, the population it will score has a date. Only the first is written into the minute. The authorisation carries no expiry, which silently asserts that the other three never move. They move.

Our position: validation is a cycle, not an event. Every AI authorisation should state a re-validation interval, name the performer and the budget, and state a decommission trigger — the condition on which the system is switched off rather than retrained. An approval with no expiry is not governance. It is a minute of a meeting.

Decay is ordinary, not exceptional.

It shows first in calibration, not accuracy, in clinical prediction, the most heavily validated applied modelling discipline there is.

Credit limits, provisioning, pricing and fraud thresholds are set on calibration and monitored on discrimination: a board shown an accuracy chart is shown the metric that fails last.

The instrument for governing an expiring accuracy is older than machine learning: the argument was never whether to recalibrate, only how often.

Drift also arrives faster in the corridor.

Nineteen per cent account growth against a five per cent contraction in service points is not the population any model was fitted to two years earlier. A vendor's home-market cadence is optimistic here by construction.

We are not arguing that models are unreliable or that initial validation was done badly, only that a validation result is dated evidence about a moving object.

The interval is easy to write and hard to fund. Re-validation needs three things in three places: labelled local outcome data, held by the operating company; access to the model or its case-level scored outputs, controlled by the vendor; and someone competent to compute calibration locally, who may be in neither. The budget line is usually in a fourth place, or nowhere. An interval nobody owns is an interval nobody keeps.

Sources and limitations. The caveats on the cards above bind: none of the four sources measures drift in a deployed commercial system, and none was written about AI re-validation. The re-validation rule is the firm's position, graded SEG-1, mandated by none of the cited evidence and unsupported by any dataset comparing authorisations with and without an expiry, because we found none meeting our standard. It is strongest where outputs set limits, prices or thresholds, weakest for low-stakes systems under stable inputs. This note is not valid as legal or regulatory advice, and nothing here suggests that a re-validated model is a correct one. Sources verified 2026-08-03.

1. The board secretary writes an expiry date into the authorisation itself and records what happens on that date if nobody acts. If the system simply keeps running, the interval is decorative.

2. The model owner names three parties in writing: who re-validates, on whose data, and out of which entity's budget. If one is unnamed, the interval will be missed for reasons later called unforeseeable.

3. The chief risk officer records the decommission trigger and the local officer who can pull it within the week. Retraining is the default answer and not always the right one; a system nobody locally can switch off is not under our control.

4. The contract owner requires dated calibration on local outcomes in place of the vendor's accuracy pack, and obliges the vendor to disclose its own retraining cadence. A model re-fitted offshore restarts the clock without telling anyone in Abidjan.