Find the delivery problem
Why AI projects fail after the demo, and what to check before rollout
An AI demonstration can work while the everyday job remains difficult. The missing piece may be client context, an exception, a handover or the team’s confidence in the result. Diagnose where the complete job breaks before adding more features. This guide gives you a practical failure review without relying on headline failure rates.
Trace a failed job from request to result, then fix the observed break and test that same job again.
Follow one job that did not work
Start with a specific request, the expected result and what happened instead. Follow the inputs, steps, decisions and handovers. Ask the person doing the work where they had to intervene or return to the old process.
Compare the live job with the demonstration. Did the demo use complete inputs, one standard client or a manually prepared source? Those choices can be reasonable for an early test, but they leave questions that rollout still needs to answer.
Avoid treating every problem as a model problem. Missing information needs an owner. A duplicate action needs a reliable integration rule. An unclear approval needs a decision about responsibility.
A mistake Said made: building before the conversation
While working with a PR team, Said saw that client copy was delaying the process. He built and trialled an AI drafting approach before having the right conversation with the people doing the work. The team pushed back; they were concerned about generic AI-generated content.
Said acknowledged that he had got the approach wrong. He then returned to the client’s perspective and asked the team what they would want if they were the client. The discussion moved towards useful personalised drafts the client could approve.
The lesson in that account is about how the change was approached. The team’s concern contained information about quality and the client relationship. A technical demonstration could not settle those questions on its own.
Look for the break that matches the evidence
| Observed symptom | What to inspect | Possible next change |
|---|---|---|
| Works for one client, fails for another | Different requirements, sources and routes. | Map the missing client variation. |
| Answers sound convincing but need rewriting | Evidence selected and claim support. | Expose sources and test unsupported claims. |
| Nobody trusts the output enough to use it | Reviewer information, quality expectations and team concerns. | Observe a real review and agree acceptance. |
| Preparation is faster, total job is slower | Checking, correction and handover effort. | Fix the part that moved work onto the reviewer. |
| Repeated events create repeated work | Integration keys, retries and persistent state. | Test a duplicate event and partial failure. |
| Information goes stale | Update triggers, owners and validity rules. | Connect approved changes to maintained records. |
Several breaks can occur in the same job. Record them separately, then prioritise the one that prevents a useful result or creates an unacceptable action. A long feature list can make the diagnosis harder to see.
Make one corrective change and repeat the job
State what the change should fix and how you will recognise that it has worked. Repeat the failed example and a nearby example that should keep working. Include the person who needs to use or approve the result.
If the job cannot be bounded or the required information cannot be obtained, change the scope or stop that approach. A useful pilot provides a decision about the work; it does not need to defend the original idea.
Assign the correction to an owner and keep a review date. Use the agent evaluation record for behaviour tests and the adoption plan for changes in daily use.
Record a diagnosis another person can check
Worked example
A report that works for the demo client only
Invented teaching example. No client outcome is claimed.
OBSERVED FAILURE
An invented live-style test requires a client-specific breakdown absent from the demo.
DIAGNOSIS
The requirement was missing from the workflow inputs.
CHANGE
Record the approved breakdown and the account owner responsible for it.
RETEST
Prepare the report for the variation and the standard client.
Check the full review effort and whether both reports meet their agreed requirements.
Further reading
Written by Said Lopez, founder of LeverageAI and author of Lean AI. Our guides draw on the method in the book and our work with clients.