Human-in-the-loop, honestly

Somewhere in most AI deployment approvals sits one sentence doing enormous work: a human reviews every decision. It appears in vendor contracts, risk registers, board minutes and regulatory filings, converting uncomfortable authorisations into comfortable ones. The decision is whether to accept that sentence as evidence of a control, or to treat it as an unproven claim about a person the board has never met, doing a job it has never costed.

Our position is blunt: a human checkpoint functions as a control only when the human holds four things at the moment of refusal — authority to say no without career consequence, time to actually look, context to see what the machine saw and what it did not, and an incentive structure that does not punish refusal. Remove any one and the checkpoint becomes a costume: it changes who is blamed after the failure without changing its probability. We call that liability theatre; it passes audits until the day it meets a loss.

Start with the law, which is more honest than most deployment plans. Article 14 of the EU AI Act does not say "put a human in the loop." It requires that high-risk systems be designed and developed so that natural persons can effectively oversee them, and enumerates what the overseer must be enabled to do.

Article 14 is a list of the conditions under which a human can refuse a machine: the legislator does not assume the human will function, it orders the system built so that functioning is possible.

Automation bias is one of the most stable findings in human-factors science. Skitka, Mosier and colleagues showed in 1999 two characteristic errors: omission, missing events the aid failed to flag even when other indicators showed them plainly, and commission, following the aid against training and fully valid contrary information. Parasuraman and Manzey's 2010 integration added that complacency emerges under multiple-task load — the normal condition of a production review queue — appears in experts as well as novices, and is not eliminated by practice.

NIST's AI Risk Management Framework — voluntary, but the reference vocabulary of most AI governance programmes — places oversight under GOVERN as an organisational design problem and treats the insertion of a human as itself a risk decision: oversight roles inherit the biases the framework catalogues.

The law demands that refusal be possible; the psychology shows it is unlikely under load unless deliberately protected; the framework makes that protection a design task, not a default. What none supplies — because it is firm judgment, not measurement — is our operational test: authority, time, context, incentive, all four, held by the named person on the worst day of the quarter. One consequence is measurable: a checkpoint that has never refused the machine is not evidence that the machine is good, but evidence that the checkpoint is absent. Override rates are a control metric, and zero is a finding.

Evidence cards

The four conditions fail predictably where our clients operate. AI deployments arrive priced on vendor-country staffing assumptions: a queue sized for a European back office, minutes-per-item ratios validated in the vendor's home market. An Abidjan operating company inherits the queue without the staffing plan behind it, and oversight lands on whoever is available, at the end of a reporting line whose real authority sits in Paris or Geneva. Each condition then degrades independently. Authority: the reviewer who refuses is overruling a system the holding company paid for, from the bottom of the organisation chart. Time: the queue arithmetic was never recomputed for local staffing. Context: the model was trained on markets with formal credit histories the corridor does not have. Incentive: throughput is measured, refusal is friction. There is also a regulatory asymmetry: the EU AI Act reaches the European holding company; no equivalent statute operates in the WAEMU jurisdictions where the system runs. Whatever oversight exists in Abidjan exists because the group built it.

Sources and limitations

Sources and limitations. The legal and framework facts rest on the text of Regulation (EU) 2024/1689 Article 14, its application timeline as amended by Regulation (EU) 2026/1744, and NIST AI 100-1; the behavioural findings rest on the peer-reviewed automation-bias literature cited above, whose laboratory provenance is flagged, not hidden. The four-conditions test is the firm's position, graded as interpretation: it is not validated as a scored instrument, and we publish no client oversight metrics because none has passed our evidence-release gate. The transferability boundary is explicit: strongest for high-risk, high-volume decisions reviewed under load; weakest for low-stakes tasks where a light checkpoint may be proportionate. This note is not valid as legal advice on AI Act compliance, nor as a claim that human oversight is useless — only that it must be built, resourced and measured before it is counted as a control.

1. The chief risk officer: publish the override rate of every checkpoint counted as a control. A rate of zero is not comfort; it is the strongest available signal that the checkpoint is a costume.

2. The operations director: recompute the queue arithmetic — items per reviewer per day against minutes of genuine review per item — on the staffing that exists, not the vendor's deck. If it does not close, the time condition has already failed.

3. The head of internal audit: trace one real refusal end to end. Who was questioned, whose metrics suffered, what changed. If none has ever occurred, return to action one.

4. The deployment sponsor: name the checkpoints that exist solely so that a person is blamable, and decide by month-end to remove or resource them. An honest answer names at least one.