Why AI pilots stall at production

A pilot proves a model can do a task. Getting an AI pilot to production proves your organisation can absorb the consequences of it doing that task every day, at real volume, with nobody preparing the data by hand. Almost no pilot is designed for the second question, which is why so many die quietly between the October demo and the January budget review. The model is rarely what fails. The substrate around it is: the pipeline, the integration, the governance, and the person who owns the workflow on the Monday after launch.

I have watched this across enough engagements to write it down as a pattern rather than bad luck. The demo worked. The rollout didn't. That is not a technology story: it is a story about what a pilot was designed to answer, and what production actually asks.

The recurring pattern

A vendor or an internal team builds a pilot in October. It demos beautifully. The model handles the workflow, the slides land, the steering committee approves moving to production. By January the pilot is dead, paused, or living in a phrase like "the next phase". Nobody quite remembers the meeting where it stopped, because there wasn't one. It degraded, and then it was quietly switched off.

Trace any of these stalls back and the failure is almost never the model. The model did the thing you asked it to do in the demo. What broke was everything the demo did not have to be true about. A demo runs once, on prepared data, watched by people who want it to work. Production runs every day, on Monday morning data, watched by nobody until it is wrong. Those are different tests, and a pilot passes the first while telling you almost nothing about the second.

Trace the stall back and it is always the same five things

Pull apart a stalled pilot and you find one or more of these. They are boring, which is exactly why nobody budgets for them.

1. The data pipeline was manual. The pilot ran on a clean export a person on the team prepared the night before. Production needs that data to flow on its own, on a schedule, with permissions, with a refresh, and with someone who owns the pager when it breaks at 6am. The pilot did not budget for the pipe, so on go-live day the workflow is starved of the one thing it needs, and a human quietly goes back to doing the export by hand. Now you have an agent and a manual step, which is worse than either alone.

2. The tool integration was a hack. The pilot used a screenshot, a copy-paste, a personal API key, or a browser extension running on one laptop. Production needs a stable interface, an audit trail, and a permissions model that survives the person who built it going on leave. The hack works right up until the builder changes their password, and then the workflow is dead and nobody can say why.

3. Governance was a Slack thread. The pilot got informal sign-off from legal in a DM. Production needs a data processing agreement, an audit trail, an escalation path, and a written policy on what the agent must not decide. The pilot did not put the general counsel and the data protection officer on the cadence, so they are reading about it for the first time at the production review, and now they are a blocker instead of a partner. That delay is not their fault. It is the pilot's.

4. Nobody owned maintenance. The pilot had a project manager who was already onto the next thing. Production needs an operating leader who owns the workflow's drift, the agent's failures, the prompt updates, and the policy changes when the business changes. There is no such person, so the workflow ages badly. Models drift, edge cases accumulate, the business shifts underneath it, and with no owner watching, it decays until someone loses trust and turns it off.

5. The pilot tested the model, not the operating system. The pilot answered "can the model do this?" The production question is "can our operating system absorb this?" Those are different questions with different answers, almost every time. A model that reads a CV well in a demo is not the same as a hiring workflow your organisation can actually run, review, and defend.

Four of those five have nothing to do with the model. The model was the easy part. It always is.

Getting an AI pilot to production is an infrastructure problem, not a model problem

Here is the reframe that changes how you budget. The gap between pilot and production is not intelligence. It is infrastructure. And the useful news is that the infrastructure is cheaper than most boards fear, because they have been quoted enterprise platform prices for what is mostly a fortnight of plumbing and a named owner.

This is roughly what the production substrate actually costs, drawn from workflows we have taken across the line.

The line that surprises people is the tooling one. Seat-based software has trained everyone to expect a five-figure annual licence for anything that touches production. A workflow platform that is compliant, auditable and self-hostable at about twenty pounds a builder seat a month is a different economic shape entirely. The expensive part of production is not the software. It is the fortnight of pipeline work and the person who owns the result, and neither shows up in a vendor quote.

If you want the fuller argument for treating this as architecture rather than a shopping list, the AI operating system piece lays out the substrate a pilot has to land in.

What pilots that ship look like

The pilots that survive to production share a small set of design choices, and they are all decisions you make on day one, not rescues you attempt at the production review. The difference is not effort. It is what the pilot was pointed at from the start.

The design choice that does the most work is the first one. A shipping pilot is the production version run at lower volume, not a separate artefact you hope to industrialise later. Name the owner, the data source, the integration, the policy and the cadence before you build, and the pilot is a rehearsal for the real thing. Name them after the demo, and you are retrofitting a foundation under a house that is already standing. That is where the cost and the delay come from.

The questions to ask before you approve a pilot

Most stalls are decided at the approval meeting, months before anyone notices the workflow degrading. The approval is where the manual pipeline and the ownerless workflow get waved through, because the demo was good and nobody wanted to be the person asking the boring question. So ask the boring questions there, out loud, before you sign.

None of these questions is about the model. That is the point. If you can answer all five before you approve, you have designed a pilot. If you cannot, you have approved a demo and scheduled its funeral for January.

The misassignment that kills more pilots than any hiring gap

When a board asks why a pilot stalled, the answer they reach for is usually "we did not have the skills". Sometimes that is true. More often it is a misassignment, and misassignment is a cheaper problem to fix than a hiring gap, which is why it is worth getting right first.

The ownerless workflow lands on IT by default, because IT owns the infrastructure it runs on. That feels tidy and it is exactly wrong. IT owns the servers and the security. It does not own the business decision the workflow makes: which candidate to shortlist, which invoice to flag, which query to escalate. So IT keeps the lights on, the workflow keeps running, and nobody with a stake in the decision is watching whether the agent is still making good calls. The workflow does not fail loudly. It drifts, and drift is invisible until it is a headline.

The fix is not more headcount. It is naming an operating leader from the business side, the person who feels it in their numbers when the workflow makes a bad call, and giving them one or two internal builders to keep it healthy. That is a role you already have in the building. This is the quiet discipline of operating leadership: owning the thing after the launch party, when the interesting work is finished and the maintenance starts.

Where the pilot actually fits, and what production looks like when it holds

Pilots are useful. They tell you whether the model can do the task and whether your team will trust the result. What they do not tell you, on their own, is whether your organisation can run the workflow. That is a different question, and it is the one that decides whether you get a system or a slide.

When the substrate is there, pilots cross to production almost without ceremony, because there is nothing left to retrofit. This is what that looks like on the other side, from a defence tech engagement where the data pipeline, the ownership and the governance were designed in from the start.

Those are not model numbers. A better model would not have produced them. They are what happens when the workflow around a competent model is designed to be owned, fed and governed. The zero at the end matters most: two months on, nothing critical had broken, because someone owned it and the substrate held. That is the difference between a pilot and production, expressed as a number.

If you want a picture of what good looks like once a workflow is live, that is the health you are aiming for, not the demo applause. And if all of this reads as too big to start, it is not. The move is to take one process, map it end to end, and design the production version of a single workflow rather than a programme. A Grain Audit does exactly that: one process, a ranked automation plan, and a plan you keep. For the wider arc this sits inside, from probe to infrastructure, see the AI operating system pillar.

AI pilots stall because the pilot proves the model can do the task and production proves the organisation can absorb the consequences. The model is rarely the problem. The data pipeline, the integration, the governance, the maintenance and the operating ownership are. Pilots that ship are designed as the production version at lower volume, with a named owner, real data, and a decision at the end.