he risk that gets the headlines is malevolence. The risk that matters is drift.
The popular fear is a system that turns hostile. Hollywood’s version, the doom. Investor-deck slide three. Politicians on Sunday morning television. None of it is the actual exposure individuals and organisations face.
The actual exposure is quieter and already everywhere. AI systems deployed without adequate governance, oversight, or grounding in consequence — optimising confidently for the wrong thing, at speed, without the feedback mechanisms that make human error correctable before it compounds.
That is not malevolence. It is institutional psychosis: the moment a system becomes so invested in its own ‘model’ of reality that it mistakes contradiction for threat, evidence for inconvenience, and decisive action for truth.
Drift Looks Like Performance Until It Doesn’t
A model or solution trained or built on yesterday’s data confidently makes today’s decisions. A pricing engine optimises for short-term margin while eroding the customer base that produced the margin. A fraud detection system slowly normalises against drifted baselines and stops flagging genuine fraud. A clinical triage assistant favours patterns it was rewarded for during training and quietly down-weights the cases that didn’t fit.
Each of these looks like performance from the inside. Throughput is up. SLA breaches are down. The dashboard glows green. The model that no one is actively correcting is doing exactly what it was asked to do.
The damage compounds in the gap between when the drift started and when someone external notices the model has been confidently wrong for months.
The Governance Gap, In Numbers
McKinsey’s 2026 State of AI Trust report finds 74% of organisations identify AI inaccuracy as a highly relevant risk. Only one-third have reached meaningful governance maturity. Lenovo’s CIO Playbook 2026, surveying over 3,000 decision-makers, finds 60% of organisations in late-stage AI adoption — but only 27% with a comprehensive AI governance framework, and 66% lacking one entirely.
The asymmetry is the story. Three-quarters of organisations have correctly identified the risk. Roughly a quarter have done anything operationally serious about it. The remainder have governance documents, working groups, framework diagrams, and a steering committee. They do not have control.
This is the exact pattern that produces drift. The model is in production. The governance is in PowerPoint.
Why Governance Maturity Lags Deployment Maturity (And Always Will)
There is a structural reason for the gap. Deployment moves at the speed of an individual team. Governance moves at the speed of the institution.
A capable team can put an AI capability into production in a sprint. Getting that same capability wrapped in a formal control framework, assigned to a named risk owner, subjected to an auditable evaluation regime, backed by documented data lineage, covered by an incident response playbook, and signed off by legal, risk, compliance, IT and security — that takes quarters. Sometimes years.
The two timelines do not converge by themselves. They diverge. Every new use case widens the gap. Every reorganisation widens it further. The institution is structurally incapable of governing what its capable individuals are deploying.
That is where drift takes root – in the ungoverned space between institutional intention and operational reality.
The Three Failure Modes Worth Naming
The literature treats AI risk as a single category. In practice from what I read it presents as three real distinct failure modes, and each requires a different control.
- Specification drift. The solution is optimising for something that no longer matches the outcome anyone actually wants. A hospital triage model optimises for throughput, not clinical risk. A policing analytics tool optimises for recorded incident history, not actual harm. The model is performing exactly as specified… it’s just the specification is wrong.
- Data drift. The solution is operating on inputs that have moved away from the distribution it was trained on. A demand forecasting system trained on 2023 patterns is now operating in a 2026 market. It is making confident predictions on data it has never seen. The model has no way to know.
- Confidence drift. The solution or tool is generating outputs with stable confidence scores while its actual reliability has fallen. This is the most dangerous one because the confidence signal — the thing the operator uses to decide whether to override — is itself the failure point. I found myself falling foul of this myself recently!
These look similar but they are different problems and most organisations seem to miss this.
What Effective Control Actually Looks Like
Control of AI systems is not control of the model. It is control of the loop the model is operating inside.
Three things need to be specified in advance, not retrofitted after something happens:
- The Consequence boundary. Where can this solution act autonomously? Where must it stop and present its reasoning to a human with the authority and the time to override? The boundary needs to be specific, measurable, and enforced by the integration — not by training. If the boundary is set by training, drift will move it.
- The Disagreement signal. How does the system tell its operator that it is uncertain in a way the operator cannot ignore? Probability scores embedded inside a JSON payload are not a disagreement signal. They are decoration. The signal needs to be loud, structured, and routed to someone with the authority to act on it.
- The Correction loop. How does an error get noticed, reported, and fed back into the solutions evaluation loop? If the answer involves a quarterly review meeting, the loop is not real. The loop needs to operate on the same cadence as the solutions use cadence — weekly, sometimes daily, occasionally in real time.
A model, solution or tool with these three things has governance. A model without them is in production unsupervised, regardless of what the framework document says.
The Cost of Late Correction
The reason this matters now is that the cost curve of late correction is non-linear.
A specification error caught in week one of deployment is a configuration change. The same error caught in month six, after thousands of decisions have been made, customers have moved, downstream systems have adapted to the solution’s output, and the operations team has built workflows around its behaviour — is a strategic problem. The solution has not just been wrong. The organisation has reorganised around the wrongness.
This is what the McKinsey survey is gesturing at when 74% of organisations report AI inaccuracy as a highly relevant risk. They are not worried about the solution being wrong. They are worried about discovering it has been wrong for longer than they noticed.
The Quiet Failure Mode Is the One to Plan For
The AI safety conversation has been captured by the dramatic. Existential risk. Sentience. Misalignment in the science-fiction sense. These are not the immediate problem. They will not be the cause of the first wave of serious institutional harm.
The first wave will be quieter. Confident systems acting on stale models of the world. Drift normalised into baseline. Governance frameworks that exist on paper while production runs unobserved. Operators trusting the dashboard because the dashboard was built before anyone thought to ask what would happen when the model stopped being right.
Uncontrolled AI is not a malevolence problem. It is a governance latency problem with high consequence and slow detection.
That is the failure mode to plan for. The headlines will catch up later.
For more on artificial intelligence, autonomy, leverage, and the irreversible changes now reshaping work and modern life, explore the other posts or subscribe.
