Why AI projects fail: seven of ten failures are data and governance, not the model
Contents
Why AI projects fail is usually not a question about the model. Seven of ten AI failures are caused by data quality and governance, not by the model. The table shows five typical causes, how each one shows up and which mechanism in the operation of an AI agent catches it:
| Cause | How it shows up | What catches it |
|---|---|---|
| No measurable task | The project is called "AI for our documents", and nobody can say afterwards whether it worked | One Vorgang (one intake, one task, one destination) and a hit rate agreed before the start |
| Data that does not fit the task | The system knows generic examples, but not the organisation's own documents and classes | Classes and examples defined with the department in week one, and a test on real documents in week two |
| Nobody owns the result after go-live | The tool runs, but nobody looks at what it did yesterday | An operator who monitors, corrects and answers for the agreed hit rate |
| Unsure cases are guessed | Errors surface weeks later as complaints | A confidence threshold below which the agent hands the case to a named person |
| Nobody can explain an assignment | An auditor or the management asks why, and there is no record | A decision log with timestamp and reasoning for every case |
Most AI failures start outside the model
The model gets most of the attention, but it is usually not the part that breaks. RAND, the US research organisation, interviewed 65 experienced data scientists and engineers about why AI projects fail (RAND, 2024). The two root causes they named most often were a misunderstood problem, with models optimised for the wrong metric, and a lack of data suited to the task. The others were technology chosen for its own sake, missing infrastructure to manage data and put models into production, and problems too hard for AI. Only the last of these is about the limits of the technology.
So the remedy is usually not in the model either. It sits in the task, in the data and in the question of who looks after go-live.
A project without one measurable task can neither succeed nor fail
The most common root cause in the RAND interviews is a problem that was never stated precisely enough. "AI for our documents" is a direction, not a task. No number shows whether it worked, so the project ends in a discussion instead of a result.
That is why we start with one Vorgang: one intake, one task, one destination. Incoming post assigned to departments is a Vorgang. So are delivery notes forwarded to the warehouse, or invoices handed to accounting with a proposed assignment. Before the Pilotbetrieb, our entry engagement, starts, we agree a hit rate with the Fachbereich, the department that owns the Vorgang: the share of cases the agent should assign correctly. Acceptance in week three is measured against that number.
Data work belongs to the task, not to a programme for the whole organisation
Data quality sounds like a project for the entire organisation. For one Vorgang it is smaller and more concrete: the classes the agent assigns to, examples for each class, and access to the documents that actually arrive.
In week one, we review real examples with the department and define the classes. In week two, we build and train the agent and test it on real documents. Gaps in the data show up in that test, on the organisation's own documents, not months after go-live.
Failures after go-live stay invisible when nobody operates the agent
RAND's interviewees describe organisations that move from prototype to prototype and are then blind to failures that appear only after a model has been deployed (RAND, 2024). A new document type, a renamed department or a supplier's new invoice layout changes what the agent sees. None of that is a model fault, and all of it erodes the result when nobody watches.
Operation means monitoring, correcting and being accountable. In the Pilotbetrieb, weeks four to seven are productive operation on real volume, and we monitor and correct. If the agreed hit rate is met, operation continues under a monthly operating agreement. What AI agent governance involves for an agent in daily use is set out duty by duty in our guide.
An unsure agent has to stop, not guess
An agent that treats every case the same hides its errors among its hits. A confidence threshold separates the two: below the agreed certainty, the agent creates no task, hands the case to a named person with its reasoning and logs the handover. Above it, the agent assigns the case and logs why.
The threshold is agreed with the department and tuned on real volume during the Pilotbetrieb. How the handover works in detail is described in our article on the confidence threshold.
An assignment nobody can explain costs trust at the first question
Projects also end when someone asks why a document went where it went, and nobody can answer. The decision log answers with one entry: what came in, where it was assigned, with what confidence, on what basis, whether a person intervened, and when.
A complete decision log is available as evidence for EU AI Act audits. The question of why becomes a lookup. Which fields belong in it is described in our article on the AI audit trail.
A stalled project restarts with a smaller goal
A stalled AI project does not need a bigger frame but a smaller one: one Vorgang with a measurable result. What the first attempt showed, which documents arrive, which classes exist and which exceptions turn up, is good groundwork for week one.
We build the agent in conversation with your department and operate it afterwards. Each of our AI employees covers exactly one Vorgang, writes into exactly one target system and hands anything it cannot assign with confidence to a person. Your team builds, configures and trains nothing itself.
Frequently asked questions
Is it usually the model's fault when an AI project fails?
Usually not. Seven of ten AI failures are caused by data quality and governance, not by the model. Typical causes are a task without a measurable goal, data that does not fit the task, and nobody watching after go-live. A better model fixes none of these. A clearly bounded task, an agreed hit rate and an operator who answers for the result do.
What should we check first in a stalled AI project?
Four questions. Is there a single task with a number that shows whether it succeeds? Who looks at what the system did yesterday? What happens to cases it is unsure about? Can a single assignment be explained afterwards? Where one of these answers is missing, that is usually where the cause lies, not in the model.
What can we reuse from our first attempt?
Mainly what you know about the Vorgang: which documents arrive, which classes exist, which exceptions turn up and who resolves them today. That is the material for the process conversation in week one. The target system stays, and so does an interface that already exists. What is new is the agent we build and the operation we take on.
How will we know it works this time?
By a number fixed before the start: the hit rate. It is tested at acceptance in week three on real documents and measured again on real volume in weeks four to seven. The Pilotbetrieb ends with a written evaluation of the hit rate achieved, cases processed and time saved. If the rate is missed, the Pilotbetrieb ends with no further obligation.
Has a project failed if the agent hands cases to people?
No. Handing over unsure cases is by design, not an error. Below the confidence threshold, the agent stops and hands the case to a named person with its reasoning. A project has failed when unsure cases are guessed and nobody knows which ones. How many cases are handed over is tuned with the department on real volume during the Pilotbetrieb.
The next step
If a first AI project has stalled in your organisation, start with one Vorgang that arrives every day and is sorted by hand today: one intake, one task, one destination. That is what the Pilotbetrieb is for. Fixed price, agreed before we start. Seven weeks, four of them productive operation on your real documents, measured against a hit rate fixed in advance.