Why Enterprise Agent Pilots Stall Before Production

Most of the numbers used to describe agent "failure" are not failure rates. They are forecasts, maturity scores, or interview-based funnels for generative AI in general, and no source we could open publishes a transparent pilot-to-production conversion rate for agentic projects specifically.

This is NKK's reading of the evidence, not a measured finding: pilots stall because six decisions stay open, one for each gate proposed below. Until they are answered, a pilot can run, demo well and never earn the funding, controls and accountability that production demands.

1. Data: what may the workflow use?
2. Ownership: who is accountable for the outcome?
3. Evidence: how will correctness be checked?
4. Integration: how does the agent act in real systems?
5. Adoption: how does the work change for the people using it?
6. Economics: what may a successful outcome cost?

Every failure rate measures a different thing

Published figures run from 2% to 62%, and each can be accurate because it uses a different verb, population and date. The MIT NANDA report, the source of the "95% get zero return" headline, is the worked example. Its own funnel reads: 60% of organisations evaluated enterprise-grade tools, 20% reached pilot and 5% reached production ^[1]. Dividing the report's figures (our arithmetic), 5% of organisations is about 25% of those that piloted and about 8% of those that evaluated. The headline uses a different denominator from the funnel. "Success" means users or executives remarked on marked and sustained productivity or P&L impact; the authors describe the figures as directionally accurate and interview-based, and the report is preliminary, not peer reviewed, and about generative AI rather than agents ^[1].

The funnel shows where the loss happens, with one caveat: it is a single preliminary report that we could read only through a third-party mirror, so treat it as one interview-based data point. The largest absolute drop, 40 points, is from evaluating to piloting. The pilot-to-production step is 15 points but loses the larger share of what reaches it (75%, against about 67%; our arithmetic). That second step is what this article is about. In NKK's reading the first loss is a prioritisation problem and the second an operating problem.

Other figures are different objects. BCG's 60% is a maturity-tier count from a survey of 1,250 senior executives and AI decision makers, assessed across 41 foundational capabilities, not a count of cancelled projects ^[2][3]. Deloitte offers the closest measured conversion: only 25% of 3,235 surveyed leaders had moved 40% or more of their AI pilots into production ^[4]. It is self-reported and covers AI broadly. Value claims need the same care. NANDA's "zero return" and Wharton's "three in four see positive returns" are not in conflict: one asks for sustained P&L-level impact on enterprise tools, the other records leaders' self-reported returns on generative AI. NANDA itself notes that individual-productivity tools are widely used without moving the P&L ^[1][5].

Build-versus-buy figures are fragile too. NANDA's interviewees reached deployment about two-thirds of the time with external partners (the report's text says about 67%, its chart 66%) and about one-third with internal builds, but the authors say this does not necessarily prove causation ^[1]. BCG finds only 11% of firms rely primarily on in-house development ^[2]. We also could not open the primary Gartner, McKinsey or S&P Global publications, so we do not use their figures.

Place each number on a ladder

The headline rates also disagree because "in production" has no shared meaning. KPMG counts "building, deploying or developing" ^[6]; LangChain's public-survey respondents report agents "running in production", with no definition of the term ^[7]; Capgemini asks about fully scaled deployment ^[8]. Forrester says three-quarters of leaders are adopting while only a small minority have agents in meaningful production, and gives no figure ^[9]. We propose a ladder that makes such claims comparable. It borrows its staged form from maturity models, including the maturity assessment in BCG's survey ^[2], and its rung wording from the surveys compared below; it has not been validated.

To use it, name the rung and the population before a statistic enters a steering-committee deck, then describe your own portfolio the same way: so many workflows at R2, so many at R3. The table places each opened statistic where its wording supports. The placements are NKK's interpretation of terse survey language and could reasonably be argued a rung either way. BCG's 60% is a laggard-tier count of organisations not achieving material value, so it sits below the ladder and cannot be placed on it.

Where pilots actually stall

Six causes recur across independent sources. Their weight varies by industry, and we know of no public dataset that ranks them by impact.

The table gives the headline evidence; the caveats and extras follow.

Data. The banking figure was up from 63% a quarter earlier ^[6]. Bain cautions that data is not clean and advises curating it as needed rather than holistically ^[11]. BCG treats data foundations, scaled AI capabilities and governance as prerequisites, and favours a few high-value workflows over everywhere at once ^[2].

Ownership. Forrester asks for every agent to be a governed identity with a named owner ^[9]. Bain suggests general managers, not the CIO, carry the targets ^[11].

Evidence. LangChain's respondents run online evaluations less often (37.3%) than offline ones ^[7]. In the Measuring Agents in Production (MAP) survey, 74% of the 31 deployed systems that answered used human-in-the-loop validation, a multi-select question; the paper's abstract words this as depending primarily on human evaluation ^[12]. Both samples are small and indicative.

Integration and identity. Forrester says the sprawl persists even after adopting the NIST AI RMF ^[9]. NIST states the benefits of agents cannot be realised without applying identity principles to them ^[13].

Adoption. NANDA ranks resistance to adopting new tools first, as a frequency rating rather than a measured impact ^[1]. In IBM's study, 83% of CEOs say success depends more on adoption than on technology ^[14].

Economics. Anthropic notes autonomy brings higher costs and compounding errors ^[15]. Forrester calls the audit burden a "trust tax" and says that cost is currently too high ^[9]. KPMG's cost-review figure was up from 61% a quarter earlier ^[6].

Is it the model or the system around it? The MAST study covers multi-agent systems on benchmark tasks, so it speaks to the integration cause, not to ownership or adoption. About three-quarters of annotated failures (our sum: 44% plus 32%) fall under system design and inter-agent misalignment rather than task verification, and the authors conjecture that better base models alone will not remove them ^[16]. Failures cluster in design and coordination, not only in model capability.

Reliability, not demos, is what production tests

A demo needs one good run. Production needs the same run thousands of times. The tau-bench authors formalised this with pass^k, the chance that all k independent trials succeed. In the 2024 paper, agents such as gpt-4o succeeded on under 50% of tasks, and in retail pass^8 was below 25% ^[17].

Treat these as launch-era baselines, not current scores. On the Remote Labor Index, the Center for AI Safety reported in July 2026 that the top model reached 15.8%, more than six times the launch figure (our arithmetic) ^[19]. That result carries caveats: the top model was given a $150 per-project budget against a $50 default, and the evaluation setup changed since launch (models now run in their vendors' own coding agents). The post does not say whether scores remain directly comparable with the launch figure, and it warns that the automated judge overstated the newest models' rates, although the 15.8% comes from human evaluators ^[19]. We did not re-check the other two benchmarks' current results, so read their numbers as dated.

These benchmarks measure autonomy, and deployed systems avoid it. In the MAP study, 68% of 60 surveyed deployed agents ran fewer than ten steps before human intervention (the abstract says "at most 10"), and 14 of 20 case studies prompted off-the-shelf models rather than tuning them ^[12]. The authors present such figures as qualitative evidence, not prevalence estimates; the sample is 86 filtered survey responses plus 20 case studies, skewed to the Americas and the authors' networks. Capability is moving fast: Stanford's 2026 AI Index reports OSWorld accuracy rising from roughly 12% to 66.3%, within about six points of human performance (the chapter's chart caption gives about 5% in 2024, so its starting point is inconsistent) ^[22]. The same chapter says agents still fail roughly one attempt in three on structured benchmarks generally ^[22]. METR estimates the task length models can complete has doubled about every seven months since 2019, measured mainly on software tasks ^[23].

NKK Production Readiness Gates

Our model is built on the prior art above, gate by gate: G1 Data on BCG and Bain ^[2][11]; G2 Ownership on Forrester and NANDA ^[9][1]; G3 Evidence on Kapoor and colleagues, tau-bench and MAP ^[10][17][12]; G4 Integration on NIST's identity work ^[13]; G5 Adoption on Deloitte and IBM ^[4][14]; G6 Economics on Kapoor and colleagues and Anthropic ^[10][15]. A pilot is scored on six gates before it earns production funding. Score each gate from 1 to 4, the numbers shown beside the options, with evidence in hand, not opinion. The funding rule is one sentence: fund only if every gate scores at least 3, and G4 scores 4 for any agent that can write to systems.

Agent identity is scored once, under G4, so the same control is not counted twice. The top level of G3 describes tracking in production, so before launch a pilot can score at most 3 on Evidence.

The weakest-link rule is NKK's judgement; no public dataset links these gates to outcomes. Weights differ by sector: a regulated bank needs G4 earlier. Two assessors may score differently, so require an evidence link for every score. The gates also assume the use case should exist at all, which they do not test.

Match autonomy to verifiability

Gates tell you whether to ship. A second model tells you how much autonomy to grant. Anthropic distinguishes workflows with predefined paths from agents that direct their own processes, and recommends the simplest solution that works ^[15]. Forrester advises bounded tasks behind approval gates with rollback paths ^[9].

Place each candidate workflow before building. In NKK's judgement, not a measured result, building automated checks to move a workflow right is often a better first investment than waiting for a better model. Placement is judgement and can be gamed to favour a preferred project; it also hides regulatory exposure, which needs its own check. As steps are added, a workflow can drift between quadrants because errors compound ^[15].

What is genuinely changing in 2026

The conditions around the pilot have shifted, even if no source yet measures the conversion rate.

Three cautions on the timeline. The adoption figure in the MCP announcement, over 10,000 public servers, is the vendor's own claim ^[24]. We found no finalised NIST agent standard, and the NCCoE paper remains a draft ^[13]. The EU Act became applicable on 2 August 2026 with exceptions, so the later high-risk dates above are the ones to plan against ^[29]. NKK's view: a later deadline is not a reason to defer the audit trail, because G4 is what production requires anyway.

KPMG's pulse gives a mixed picture. Governance confidence rose, with 73% of leaders confident in the governance and capabilities in place (57% a quarter earlier), while the share building controls into agents fell from 43% two quarters ago to 30%, which KPMG attributes to many organisations relying on guardrails already in place ^[6]. The survey is self-reported by 314 US leaders, and we found no stated denominator for the 30%. Our interpretation, not KPMG's: building may be outpacing controls built into agents, but the wider data do not show that the gates are being skipped.

Sequencing matters as much as speed. BCG's Amanda Luther, speaking of agentic AI, put it this way:

What to do next

1. Name your rung. Classify every agent workflow on the ladder and report the portfolio, not a single "in production" number.
2. Write a one-page evidence plan before building. Include the held-out tasks, number of repeated trials and human review sample.
3. Name owners and issue identities. One business owner and one technical owner (G2), and one least-privilege identity per agent (G4) ^[9].
4. Set a cost ceiling and a stop rule. Decide the maximum cost per successful outcome and the date at which a pilot ends if it has not reached 3 on G3.
5. Start bounded, behind approval gates. Place the workflow on the map and widen autonomy only as controls prove themselves.
6. Rehearse the failure drill. Test rollback and revoke an agent's credentials before launch.
7. Put the J-curve in the business case. Economists show intangible investments make measured gains lag adoption ^[30]; that work is about general-purpose technologies (software, computers and AI generally), not agents specifically, so treat it as framing.

Conclusion

On the evidence we could open, agent pilots stall less because agents do not work and more because organisations leave six decisions open: which data, who owns the outcome, how correctness is checked, how the agent acts in real systems, how the work changes, and what a result may cost. The published rates cannot settle that for you, and for agents specifically they do not yet exist in a form that deserves trust. Score your own pilots against the gates, state your own rung, and track and share your own conversion rate internally. It will be more useful than any figure in this article.