Blogs
Engineering

AI promised faster development. Most companies got faster chaos instead.

AI did exactly what it promised: it made writing code faster. The thing everyone expected to follow — faster delivery of working software — was never something the tool could deliver on its own.

Eli HeakinsJune 30, 20266 min read

AI tools are now nearly universal in software development — and yet a widely cited survey concluded that 95% of enterprise AI initiatives produced no measurable bottom-line impact. The reflexive read is that AI was oversold. We think that’s the wrong lesson. The promise that was made — faster code generation — has largely been kept. The promise that was heard — that more code would result in more business value — has not yet been realized. That gap is the productivity paradox. Arsenal AI’s perspective isn’t that significantly less time should be spent writing code; it’s that the time needs to be reallocated — from writing code to the technofunctional collaboration that refines the business outcomes and the technical considerations to account for up front, so that scalable, enterprise-grade code is what gets produced.

The problem

AI made writing code fast. The work didn’t vanish — it moved.

For decades, building software meant mostly one thing: typing the code. Now a machine does most of that. The hours don’t disappear — they have to be re-spent on the two jobs the machine can’t do: deciding what to build, and checking that what got built is right.

Before AI
Decide
Write the code
Review
the same hours, redistributed
With AI
Decide what to build
Write the codeAI
Review what it wrote
Decide what to build

Turn a fuzzy goal into a precise plan. AI builds exactly what you describe, so getting this right matters more than it ever did.

Write the code

Translate the plan into working software. This is the step AI now does most of, and quickly — the part that used to dominate the day.

Review what it wrote

Read and approve each batch of changes — a pull request, or PR — before it ships. AI writes these faster than people can check them.

Where engineering effort goes, before and after AI. Gold is the work AI absorbed; the hours it freed don’t leave the system — they move to the human judgment at both ends.
The paradox

Everything moved except the thing that mattered

What AI sped up — generation
Tasks completed
+21%
Pull requests merged
+98%
The cost that came with it
PR review time
+91%
PR size
+154%
Unreviewed merges
+31%
Incidents per PR
+243%
~0%

What was supposed to follow — company-level throughput, delivery, and quality — showed no significant change.

Fig. 1 — Change at high AI adoption, from a study tracking engineering activity across more than 10,000 developers (Faros AI, 2025). Gold is generation; red is the cost that came with it.
The mechanism

Speed up one step, and the bottleneck just moves to the next.

The two constraints cheap generation exposes were always there, hidden behind the cost of typing: deciding what to build, and confirming that what got built is correct. Call them direction and verification. Neither is performed by typing, so neither got faster when typing did — and that’s where the unconverted velocity piles up and dissipates.

The shift

Optimizing one part of a process doesn’t produce holistic efficiency unless every step scales evenly.

The AI code-generation lifecycle

Only the middle step got cheap.

Human judgment
Direction
Define business outcomes and technical requirements.
Costunchanged
Automated by AI
Generation
AI writes the code to those requirements.
Cost↓ now cheap
Human judgment
Verification
Confirm the result meets those requirements.
Costunchanged
↺ each release feeds the next decision
Fig. 2 — AI made the middle of the lifecycle cheap and left the two ends untouched. Both ends are human judgment, not typing, so that’s where the work now collects.
The expensive part

Deciding what to build is now the expensive part

Start with direction, because it comes first: an error here wastes everything downstream, including the scarce review capacity you’ll spend checking work that should never have been built.

Here’s the uncomfortable arithmetic. If your team ships at some velocity, and only a fraction of what you ship actually delivers value, then raising velocity raises the absolute amount of wasted work in exact proportion. And that fraction is low even at disciplined firms. The cleanest numbers come from controlled online experiments: at Microsoft, only about a third of well-designed ideas improved the metric they were built to move; a third did nothing; a third made things worse. In heavily optimized products, success rates fall to 10–20%.

Cheap generation doesn’t change those odds. It just buys more tickets in the same lottery — and the losing tickets pile up faster too. The faster you can build, the more it matters that you’re building the right thing.

Direction

AI deals more hands. It doesn’t change the odds.

Before AI9 well-built ideas shipped
With AI27 well-built ideas shipped, same hit rate
Improved the target metricNo effectMade things worse
Fig. 3 — In controlled experiments, only about one in three well-built ideas improves the metric it targets; a third do nothing, a third make things worse (Microsoft). Faster building multiplies the volume, not the win rate — so the losing bets pile up just as fast.

There’s a compounding effect. Code is now generated from natural-language specs, so the spec has become the dominant input. AI faithfully amplifies whatever you hand it. A sharp spec yields good software at scale; a vague one yields wrong software at scale. Benchmark studies bear this out: a large share of task specifications turn out ambiguous enough to need repair, and fixing them sharply raises success rates, while the models themselves rarely notice the ambiguity or ask for clarification. Writing that spec has quietly become the highest-leverage job in the pipeline, and it’s the one AI helps with least: it’ll draft your requirements and mock-ups, but the judgment about what’s worth building is left exactly where it was.

Verification

Verification doesn’t break loudly. It erodes quietly.

Now the downstream constraint. Reviewing well has a surprisingly low ceiling: defect detection collapses past roughly 200–400 lines per review, and an AI agent can emit several thousand lines in a sitting. The reviewer can’t expand to match.

So what happens? Not an infinite queue. Real teams have a relief valve, and it’s worse than delay: when review capacity is overwhelmed, people cope by reviewing less — approving bigger changes with less scrutiny, waving work through unread. The constraint is satisfied on paper while its actual function quietly fails. That’s why the symptom isn’t ever-lengthening cycle times; it’s declining quality. The field data show exactly this: as adoption rises, review time, defect rates, and the share of changes merged with no review at all all climb together. The bottleneck doesn’t announce itself. It hides as erosion.

Independence

You can’t just point AI at the problem

AI code review tools were perceived to be the solution; however, they haven’t driven any efficiencies. They’re worth using for the obvious stuff. But they can’t be the whole answer, for a reason that’s easy to miss. An AI reviewing AI-generated code shares the training distribution, and therefore the blind spots, of the model that wrote it. Its errors are correlated with the very errors it’s supposed to catch. Automated review raises the floor; it can’t supply independent judgment.

Independent judgment has to come from somewhere the generator can’t reach. In practice that means leaning harder on staged, real-world exposure: canary releases, ring rollouts, restored beta cohorts. This is verification whose capacity scales with users rather than reviewer attention, and which is genuinely independent of the model because real traffic exercises paths no reviewer or sibling model would anticipate. It catches what review can’t, and it doesn’t consume the one resource that won’t scale: human cognition. (What it can’t catch is the software that works perfectly and still shouldn’t have been built — which brings you back to direction.)

Precedent

None of this is new

This is the information-technology productivity paradox repeating. In 1987 Robert Solow quipped that you could see the computer age everywhere but in the productivity statistics. The resolution, established over the following decades, was that general-purpose technologies pay off only alongside slow, expensive, hard-to-measure investments in reorganizing the firm around them. The technology shows up in the office long before it shows up in the numbers; the lag is the time it takes to rebuild. Measured productivity dips before it climbs: a J-curve. AI is early on that curve.

Which points at the real lesson. The firms pulling ahead aren’t the ones with privileged access to models; everyone has the models. They’re the ones rebuilding the human layer of the organization around what the models changed: more capacity for deciding what to build, more for confirming it works, less devoted to the construction that used to be the hard part. The ratio the industry inherited from the era when writing code was slow is being re-derived from scratch. The teams that do that on purpose will look — and perform — very differently from the ones waiting for it to happen to them.