Every AI agent demo works. That is the problem. A demo runs one path, on data someone cleaned that morning, in front of an audience that wants it to succeed. Production runs every path, on data nobody owns, in front of users trying to get their actual job done.
The numbers bear this out. Roughly 17% of organisations have deployed AI agents, while more than 60% expect to within two years — one of the steepest intended-adoption curves the analyst surveys have recorded. The gap between intent and deployment is where most budgets currently sit.
We build agent systems for a living, so this is not a warning about AI being overhyped. It is a fairly specific claim: the things that kill agent projects are almost never the model. They are the four problems below.
1. Non-deterministic output breaks everything downstream
Around 70% of leaders name non-deterministic output as their top production-readiness barrier, and it is the correct thing to worry about. Traditional software has a defect rate you can drive toward zero. An LLM has a distribution. The same input can produce a good answer on Monday and a subtly wrong one on Thursday, and "subtly wrong" is worse than "failed" because nothing alerts.
The teams that ship treat this as a design constraint rather than a bug to be fixed by prompt tuning. In practice that means:
- Constrain the output space. An agent that returns one of nine enum values is testable. One that returns prose is not. Push structure as far down the stack as you can — tool calls with typed arguments, not free text you parse later.
- Build the eval set before the agent. Fifty real cases with known-good answers, collected from actual tickets or transcripts. Without this you cannot tell a prompt improvement from a coin flip, and you certainly cannot tell whether a model upgrade helped or quietly regressed a category.
- Decide what "wrong" costs. An agent drafting an internal summary can be wrong 5% of the time. One issuing refunds cannot. That number should drive the architecture, not the other way round.
2. Integration is the actual project
46% of organisations cite integration with existing systems as their primary deployment challenge, and in our experience that number understates it. The agent logic is usually a few hundred lines. The work is everything around it: the CRM that has three fields meaning "customer status", the API with no staging environment, the permissions model that assumes a human is behind every request.
That last one deserves attention. An agent reaching into five systems inherits the union of their access, which is rarely what anyone intended. If your agent authenticates as a service account with broad scope, you have built a way for a prompt injection to read anything that account can read. Scope agent credentials to the narrowest set that makes the use case work, and assume the agent will one day be persuaded to do something you did not plan for.
3. Governance arrives after the incident, not before
The most uncomfortable statistic in the 2026 data: a large majority of firms running agents in production still lack a formal governance model. Only about one in five has a mature one.
Governance sounds like a committee, but the version that matters is small and technical:
- Every agent action is logged with its inputs, including the retrieved context — not just the final output. When something goes wrong, "what did it see?" is the only question worth asking, and you cannot answer it retroactively.
- A spend ceiling per agent, per day. Runaway cost from continuous operation is a documented failure mode. An agent in a retry loop against a paid API is a bill, not an outage, so nothing pages anyone until finance notices.
- A named owner. Not a team — a person who is accountable for what it does.
4. The use case was chosen to demo well
One of the most common mistakes is picking a use case that looks impressive in a demo but is difficult to operationalise. "An agent that answers any question about our business" demos beautifully and cannot be evaluated, scoped, or safely permissioned. "An agent that drafts a first-pass reply to refund requests under a set value, for a human to approve" is dull in a boardroom and ships in six weeks.
The heuristic we use when scoping: a good first agent has a narrow blast radius, a human checkpoint, and an obvious metric. If you cannot name the number that goes up, you are building a demo.
What a realistic first agent looks like
For a company starting now, the shape that works:
- One workflow, one system of record, one team as users
- Human approval on anything that writes, emails, or moves money
- An eval set of 50+ real cases, run on every prompt or model change
- Structured output, logged inputs, a daily spend cap
- A defined kill switch that someone has actually tested
That is a smaller project than most AI strategy decks propose, and it has a far higher chance of being in production next quarter. Once one agent is running and observable, the second is mostly reuse — the logging, evals and permissions work is the part that carries over.
Common questions
How long should a first agent take to build?
For a scoped workflow with one system integration, six to ten weeks is realistic, and most of that is integration and evaluation rather than model work. Projects that run longer usually expanded scope mid-flight rather than hitting a technical wall.
Do we need to fine-tune a model?
Almost never at the start. Retrieval quality, prompt structure and tool design account for most of the achievable gain, and they are far cheaper to iterate. Fine-tuning is worth revisiting once you have an eval set proving a specific, repeated failure that context cannot fix.
What is the most common reason a pilot stalls?
No owner and no metric. Technically sound pilots quietly stop when the person driving them changes priorities and nobody can point to a number that would drop if it were switched off.
Where to start
If you are somewhere between "we should do something with agents" and "our pilot has been nearly ready for four months", the useful next step is usually a scoping conversation rather than a build. We do AI strategy consulting for exactly that stage, and agent development when the shape is clear. If you would rather just talk it through, get in touch.