The 5% of AI that works

Ninety-five percent of what you read about AI this year was written to be read, not used. Content about content. A screenshot of a thing that worked once, on a Tuesday, in a demo, with the lighting just right. A demo is a promise. Deployment is the argument.

The loud ninety-five

The loud ninety-five changes everything every fortnight. It will replace the entire department. Then it will make the department ten times more productive. Then the department will become a collection of agents reporting to a founder who appears to have purchased a very expensive microphone. The language gives it away. "AI-first."

"Autonomous." "Ten times." "Zero-touch." "Replace your workflow." Most of these describe a posture rather than a use. The interesting thing about a demo is that the environment has been chosen to make the idea succeed. The interesting thing about deployment is that the environment has not. A real workflow has incomplete information.

Deadlines. Politics. Legacy systems. Permissions. Exceptions. Bad data. Unclear ownership. A person who has been doing the job for seven years and has learned fifteen tiny rules nobody wrote down. The demo gets the happy path. The organisation is mostly everything around it.

The boring five

The useful five percent is quieter. It is the same few tasks done reliably because somebody did the unglamorous setup once. A draft that arrives already shaped. A first pass on the tedious thing, not the final word on it. A recurring summary. A classification task. A check that catches the wrong number before it reaches a client.

A system that knows which source is authoritative. A boundary that prevents an irreversible action without a human decision. None of this makes a very exciting thread. All of it can save a lot of time. The five percent is useful for the same reason good operations is useful. It reduces repeated friction.

The prompt is not the system

A lot of AI work still starts and ends with prompting. Ask it better. Give it a role. Add context. Tell it to think harder. Those things can improve an answer. They do not create a reliable operating environment. Once AI is used repeatedly, different questions become more important. Where does the context come from? Which source wins if two sources disagree?

What information is missing? What is the expected output? How is quality checked? What may the system do automatically? What always requires a person? Who owns the workflow when it fails? How will anybody know whether it is saving time or quietly creating a new class of work? That is not prompt engineering. That is operating design.

Bad process does not become good because it has an agent

This is where a lot of AI programmes become expensive. Take a workflow nobody fully understands. Add AI. The result is not transformation. It is a faster route through the ambiguity. If people disagree about how the work should happen, the model needs some version of that disagreement resolved. If the source data is unreliable, the model has no magical route around the reliability problem.

If ownership is unclear, automation can make it less clear by moving more of the activity away from the people who used to notice the gaps. If the organisation cannot define the user problem, the roadmap becomes a collection of technically impressive things somebody was able to build. AI does not remove organisational debt. It often makes it easier to automate it.

Map the work before automating it

Before automating a workflow, I want to understand at least four things.

What actually happens now?

Not the policy. Not the onboarding deck. What happens when the work hits Tuesday afternoon and somebody needs the answer.

Where does context enter?

Which documents? Which systems? Which conversations? Which experienced person's memory?

Where does judgement happen?

Which decisions are genuinely rules? Which require interpretation? Which have consequences the system cannot own?

Where does failure currently appear?

Does it get caught? Does somebody compensate for it manually? Does the customer find it? Does another department quietly absorb the cost? Without that map, automation risks removing the human friction that was also functioning as the quality check.

Data is not plumbing

AI programmes often treat data as the boring prerequisite beneath the interesting work. It is the work. If a system is meant to understand projects, but project information is inconsistent across five tools, the AI problem is partly an information architecture problem. If client knowledge lives in DMs and calls, a model cannot retrieve what was never captured.

If half the project statuses mean different things to different teams, adding intelligence on top does not create shared truth. The phrase "single source of truth" deserves some of the suspicion it gets. But the underlying need is real. A system can only reason over the organisation you have represented. If the representation is fiction, the intelligence on top becomes very articulate fiction.

Ownership beats novelty

Someone needs to own the product. Not merely build it. Not merely sponsor it. Own the problem it exists to solve. That means deciding: Who is it for? What is it meant to improve? How will value be measured? What will it deliberately not do? Which dependencies have to exist first? What happens when users reject it? What gets removed if it works?

A backlog is not a mandate. A demo is not adoption. A technically functioning system is not an organisationally useful one.

Adoption is part of the product

One of the stranger assumptions in technology projects is that adoption arrives after the product. Build the thing. Then train the people. Then wonder why they continue doing what they were doing yesterday. Adoption begins much earlier. Does the system solve a problem the user actually has? Does it fit where the work already happens?

Does it reduce effort or simply move effort? Does the user trust the output? Can they understand when it is uncertain? Does using it create another administrative obligation? A tool that saves ten minutes and adds fifteen minutes of verification has not created a productivity gain. A tool that works beautifully but requires everyone else to reorganise their day around it has a change-management problem disguised as a product.

Human in the loop is not a magic phrase

"Human in the loop" gets used as if the presence of a person automatically makes a workflow safe. It depends where the person sits. If a human is asked to check hundreds of machine-generated outputs at speed, they may be in the loop technically and absent cognitively. If the system presents uncertain information with the same confidence as reliable information, the human has been given a poor decision environment.

If nobody knows what the person is actually accountable for checking, the phrase has solved nothing. The useful question is: Where does human judgement materially improve the outcome, and what information does that person need to make the judgement well? Sometimes that means approval. Sometimes review. Sometimes escalation. Sometimes the correct design is not to automate the decision at all.

The measurement problem

AI work is unusually vulnerable to measuring activity instead of value. Number of prompts. Number of agents. Number of tasks automated. Hours theoretically saved. These can all be interesting. None proves the organisation is better. I would rather know: Did the work get faster? Did quality hold? Did error fall? Did the human effort actually reduce?

Did customer experience improve? Did the same work simply move into checking, correcting and maintaining the new system? Did the organisation stop doing anything because the AI now did it? If the answer to the final question is always no, some of the productivity story may be imaginary.

The tell

The useful part was never the magic. It was the discipline around it. The setup. The constraints. The source. The ownership. The workflow. The check before anything ships. The boring parts nobody screenshots, which is precisely why they are easy to skip. The five percent is quiet for the same reason it works. It has survived contact with the actual organisation.

The question is not whether AI works. The question is whether you have built the conditions in which a useful part of it can work repeatedly. Most organisations are still spending more time looking at the demo than designing those conditions. That gap is the opportunity.