I have watched capable CTOs greenlight AI projects that were dead on arrival. Not because the technology failed — the models were fine — but because the project had been designed, from the first meeting, in a way that made success impossible.
The failure rate gets quoted constantly and the number moves depending on who is selling what. The number matters less than the pattern, and the pattern is remarkably consistent: the fatal decisions are made in planning, before any code exists. Which is good news, because planning is cheap to change.
Here are the five I keep seeing.
1. Starting with the technology instead of the problem
The fastest way to kill an AI project is to begin with "we should be doing something with AI." That is a solution wearing a problem's clothing, and it produces projects optimised for how impressive they sound in a board update rather than for whether anything gets better.
The tell is that nobody can state what success looks like in hours or pounds. If the answer to "how will we know this worked" is "we'll have AI capability", the project has no success condition and therefore cannot succeed.
A counter-example. A client's agronomists missed their bonus one Thanksgiving because rebate reporting couldn't be reconciled and confirmed in time. Finance and the agronomists spent days on manual reconciliation — days those agronomists would otherwise have spent with customers. Nobody in that story wanted AI. They wanted their bonus on time. That is a real problem with a countable cost, and it made a far better first project than anything that started with the technology.
If you cannot quantify the problem, you cannot measure whether your AI solved it — which means you will never be able to fund phase two.
2. Not knowing how bad your data is
Most organisations do not have data. They have decades of accumulated spreadsheets, inconsistent formats, duplicate records, and systems that disagree with each other. Then that gets fed to a model and everyone is surprised when the output is confident nonsense.
The planning error is one of allocation. Teams typically budget something like a fifth of the timeline for data work. In my experience it is consistently the majority of the project — often two thirds or more.
On one rebate system, the apparently trivial task of matching product codes across an ERP, a warehouse system and supplier feeds took three months. The same herbicide appeared as ROUNDUP PRO BIACTIVE 360 — 20L from one supplier, Rndup ProBio 360g/L Glyphosate 20 Litre from another, and GLY 540 from a third. A human agronomist spots those as the same product instantly. Naive string matching treats them as three products, and every rebate calculation downstream is then wrong.
The fix is unglamorous: audit before you commit. Run a two-week discovery sprint, pull samples from every source system, try to merge them by hand, and count the exceptions. If a meaningful share of records have quality problems, your first AI project is the data cleaning. That is not a delay to the roadmap. That is the roadmap.
3. Building the Death Star
"Let's automate invoice processing" becomes "let's build an intelligent financial platform that integrates with everything and transforms how we work." That is not ambition, it is scope creep with a strategy deck attached, and it guarantees that nothing ships before patience runs out.
The antidote is deliberately unambitious. One client had dozens of candidate use cases and found the choice paralysing. We picked something small and deliberately outside the core line of business, so that failure would embarrass nobody: a single-purpose document reviewer. Six weeks. It took a process that ran in twenty-four hours at about £60 a task down to under three minutes at about 10p.
The point was never the cost saving. The point was that a sceptical business had now seen it work on their own data, which is what funded everything after it.
Four criteria for a first project:
- Well defined — one process, clear inputs and outputs
- Contained — failure cannot break anything critical
- Measurable — you can state the before and after in numbers
- Expandable — success points somewhere obvious next
4. Treating AI like software
Traditional software is deterministic. You write it, test it, deploy it, and it behaves the same way in September as it did in March. Models do not. They degrade as the world moves and the training snapshot doesn't. A system that was 98% accurate at launch can be materially worse six months later, and — this is the dangerous part — it will not tell you. It carries on producing confident output at a steadily worse error rate.
On a call analytics system that categorised customer service calls and flagged potential complaints and vulnerable customers, we saw exactly this: new products and new terminology entered the conversation, and classifications drifted. The fix was to route flagged calls back as retraining data and monitor confidence scores for drift, so corrections compounded rather than evaporating.
Design for this on day one: log every decision and its inputs, review low-confidence predictions by hand, track performance weekly rather than quarterly, and make corrections flow back into training automatically. If your project plan has a "deploy" milestone with nothing after it, you have built something with a shelf life.
5. No human in the loop
The fastest way to destroy trust in an AI system is to let it make consequential decisions unsupervised in its first six months. You need one bad outcome, visible to the business, and the political capital for the entire programme is gone.
The rebate assistant we built could analyse thousands of transactions and recommend which products a salesperson should prioritise. It never sent anything to a customer. It surfaced the analysis to the agronomist, who combined it with what they knew about that farm — soil, history, relationship. The system did the arithmetic; the human supplied the judgement.
That is not timidity. It prevents catastrophic errors, it builds trust at a rate the organisation can absorb, and every human correction is a training example you would not otherwise have had.
The first-win checklist
Before you start, all five should be true:
- The problem is specific and its cost is quantified
- The data has been audited, and is clean enough — or cleaning it is the project
- The scope is small, contained, and low-risk if it fails
- A human reviews consequential decisions
- Monitoring, logging and retraining are designed in, not deferred
If you cannot tick all five, you are not ready. That is a perfectly reasonable place to be, and considerably cheaper than discovering it eight months in.
None of this is about having access to better technology. Everyone has the same models. The difference is discipline, a narrow first target, and a willingness to do the unglamorous data work that makes the interesting part possible.