A pilot proves a model can do a task. Production proves an organisation can absorb the consequences. Most can't, yet, and the gap between the two is where AI programmes go to die quietly.
We have seen the same pattern across enough engagements to write it down. The model is rarely the problem. The pilot was a clever solution to the wrong test.
The recurring pattern
A vendor or internal team builds a pilot in October. It demos beautifully: the model handles the workflow, the slides land, the steering committee approves "moving to production." By January, the pilot is dead, paused, or "in the next phase." Nobody quite remembers when it happened.
Trace it back and you find one or more of these:
1. The data pipeline was manual. The pilot ran on a clean export prepared by a person on the team. Production needs the data to flow on its own, on a schedule, with permissions, with refresh, with someone owning when it breaks. The pilot didn't budget for the pipe.
2. The tool integration was a hack. The pilot used a screenshot, a copy-paste, a personal API key, or a browser extension. Production needs a stable interface, an audit trail, and a permissions model that survives the person who built it leaving.
3. Governance was a Slack thread. The pilot got informal sign-off from legal in a DM. Production needs DPA, audit, escalation paths, and a written policy. The pilot didn't include the GC and DPO on the cadence, and now they are reading about it for the first time at the production review.
4. Nobody owned maintenance. The pilot had a project manager who was already on to the next thing. Production needs an operating leader who owns the workflow's drift, the agent's failures, the prompt updates, the policy changes. There is no such person, so the workflow degrades and gets quietly switched off.
5. The pilot tested the model, not the operating system. The pilot answered "can the model do this?" The production question is "can our AI operating system absorb this?" Different question, different answer, almost always.
The model was the easy part.
What pilots that ship look like
The pilots that survive to production share a small set of design choices.
- They define the production version before the pilot starts. The owner, the data source, the integration, the policy, the cadence are all named on day one. The pilot is the production version at lower volume, not a separate artefact.
- They run on Monday morning data, not curated demos. Real volume, real noise, real edge cases. If the model cannot handle the actual inbox, no demo will save it later.
- They have an operating leader, not just a project manager. The person who will own the workflow on the Monday after launch is the person running the pilot.
- They are short on purpose. Weeks, not quarters. A short timebox forces real questions early.
- They end in a decision. Ship, kill, or budget the infrastructure to scale. Pilots that end in "let's keep iterating" are pilots that never end.
The five questions to ask before approving a pilot
Before you sign off on the next AI pilot, ask:
- Who owns this workflow on the Monday after launch?
- Where does the data come from, on what schedule, with whose permission?
- What tool calls does it make, and how are they logged and reversed?
- What is the written policy on what the agent must not decide?
- What forum reviews this workflow's drift, and how often?
If any of those answers is "we'll figure it out later", the pilot was wished into existence rather than designed.
Where pilots fit in the larger arc
Pilots are useful. They tell you whether the model can do the task and whether your team will use the result. What they do not tell you, on their own, is whether the organisation can run the workflow.
That is what the AI operating system is for. The pilot is a probe. The AI OS is what the probe lands in. If the substrate is not there, no pilot will reach production. If the substrate is there, pilots cross to production almost automatically.
For the longer arc, see from AI experiments to AI infrastructure. It covers when to make the switch and what to switch into.
A working summary
AI pilots stall because the pilot proves the model can do the task and production proves the organisation can absorb the consequences. The model is rarely the problem. The pipeline, the integration, the governance, the maintenance, and the operating ownership are. Pilots that ship are designed as the production version at lower volume, with a named owner, real data, and a decision at the end.
Common questions
- Why do most AI pilots fail to reach production?
- Because the people building the pilot are usually the people incentivised to declare it a win, not to fund what comes after. A vendor closes when the demo lands. An internal champion moves to the next priority. Nobody in the room is paid to ask whether the pipeline, the integration, or the governance will survive real volume, so those questions get asked in January instead of October, once the budget and the sponsor have both moved on.
- How do you design an AI pilot that actually ships?
- Get the production infrastructure costed and approved in principle before the pilot starts, not after it demos well. Put that budget on the same approval as the pilot budget. If the answer to 'what would this cost to run properly' is still open once the pilot succeeds, you have built a demo with good PR, not a pilot.
- What is the most common single reason pilots stall?
- It usually isn't a hiring gap, it's a misassignment. Boards default to handing the ownerless workflow to IT, because IT owns the infrastructure it runs on. But IT doesn't own the business decision the workflow makes, so it can keep the lights on without anyone actually watching whether the agent is still making good calls. The fix is naming an operating leader from the business side, not adding headcount to IT.
When reading turns into doing
The Grain Audit maps one People Ops process end to end, ranks the highest-return automations, and hands you a 90-day plan you keep whether or not we work together.
Two weeks. GBP 2,000, credited in full against a programme. Three slots a month.
Book a Grain AuditIf this resonated, there's more.
Subscribe to receive new Intelligence pieces as they're published. No noise, just the work.
By subscribing you agree to our Privacy Policy. Unsubscribe any time.
