AI assistants write code quickly.However, that this automatically leads to faster projects is not a given.A Harvard analysis picked up by t3n concludes that the productivity effect of AI tools largely evaporates in practice – and not when generating, but when reviewing the result.

The reasoning is understandable for anyone who has ever taken over someone else's code: what a machine produces in seconds must still be read, understood, tested and accounted for by a human.The bottleneck thus shifts from production to review.

The bottleneck moves, it does not disappear

In classic development processes, writing code is only part of the work.It is joined by coordination, architecture decisions, tests, code review – i.e. the collegial review of changes before they are merged into the main version – as well as deployment and operations.

If AI tools now deliver significantly more code suggestions in a shorter time, the volume that must pass through this review path grows.More lines mean more review effort.And review is harder to parallelise than code generation because it requires contextual knowledge of the project.

There is also a qualitative difference: your own code is understood while writing it, generated code must be reconstructed afterwards.That costs cognitive work that often does not appear in effort estimates because it simply did not exist before.

What this means for project budgets

For clients, the consequence is uncomfortable but useful: a tool alone does not lower costs.What matters is whether the surrounding process can handle the additional output.

In practice, this means clarifying a few questions early in project planning:

  • Who reviews generated code, and with what time budget?
  • What test coverage exists so that errors are caught automatically instead of manually?
  • Are there binding rules on which parts of the code may be created with AI support at all – for example not in safety-critical or heavily regulated areas?
  • How is it documented which parts of a system were created how?

Where these questions are answered, AI support can take over routine work.Where they remain open, a backlog builds up: plenty of material, little verified substance.

A second perspective: LLMs in operations

That the benefit depends heavily on the use case is shown by a practical test by Golem.de on the topic of monitoring.Monitoring providers advertise AI-supported root cause analysis – i.e. the automatic identification of the actual error cause behind an incident – sometimes with very large promises.

The test explores the question of how far you can get on your own if you link a large language model (LLM) directly with log excerpts, i.e. with the log files that systems generate in operation.The approach is deliberately down-to-earth: no product promise, but a verifiable application on real data.

Both sources show the same pattern from two directions.The Harvard analysis describes why blanket productivity promises for code writing do not hold up.The monitoring test examines where a concretely delimited use case actually delivers.What both have in common is scepticism towards the automatism that using AI equals time gained.

The difference between generating and interpreting

An interesting point: in troubleshooting, the model's task is different from that in code writing.It does not produce an artefact that permanently remains in the system and must be maintained, but helps to sort and interpret existing information.

Misinterpretations become apparent more quickly in this scenario because the model's hypothesis can be checked immediately against the running system.In code, by contrast, defects can remain undiscovered for a long time and only become visible in operation or with the next change.

From this, a cautious rule of thumb can be derived: AI support is most reliable where the result can be verified immediately and cheaply.The more expensive the review, the smaller the net gain.

Recommendations for project practice

For ongoing and planned projects, some steps can be derived from this:

  1. Assess AI use not across the board but per task type – boilerplate, tests and analysis work behave differently from core logic.
  2. Plan review capacity in advance instead of treating it as a residual quantity.
  3. Expand automated tests and static code analysis so that review scales.
  4. Measure instead of believe: compare lead times and error rates before and after introduction.
  5. Test providers' product promises against your own data before committing licence budgets.

Assessment

For web and software projects, this means: AI changes the distribution of work, not its total amount – at least not automatically.Anyone who wants to save budget invests first in quality assurance, test coverage and clear review processes, because that is precisely where it is decided whether faster-generated code also goes into production faster.

And anyone who buys AI functions from providers should test them on their own, clearly defined use case before making them the basis of project planning.

Sources