How much work do AI agents actually take on – and where does their influence end? A research team evaluated 769 task logs from the development of its own AI model. Agents are defined here as systems that not only provide individual answers but work on tasks independently across multiple steps.

The result can be summed up in two figures. Up to 55 per cent of the method proposals came from AI agents. At the same time, humans made over 85 per cent of the final decisions. The systems were therefore heavily involved in preparation, but hardly at all in the final determination.

More activity is not more autonomy

The authors of the study warn against precisely this confusion: a rising number of agent contributions does not automatically mean that the systems are acting more independently. Anyone who reads the volume of generated proposals as a measure of autonomy overestimates the actual level of maturity.

In practice, this distinction is more than a semantic nuance. It determines how you assign responsibilities in projects, which approval steps you plan for, and how you present the benefits of AI tools in reporting.

The overlooked effect: tasks that would otherwise be left undone

Another finding deserves attention: around a third of the evaluated tasks would not have been tackled at all without AI. The gain here is therefore not that existing work is completed faster, but that additional work takes place at all.

These are typically things that get pushed to the back burner in everyday work: working through an alternative implementation variant, checking an edge condition, preparing a data situation for which there is otherwise simply no time. Such tasks do not appear in classic efficiency calculations because there is no before-value to compare against.

What this means for evaluating AI use

Many cost-benefit analyses of AI in development environments ask about hours saved. The evaluation suggests that this perspective is too narrow. It makes more sense to consider several angles side by side:

  • Preparation output: How many robust options are on the table before a decision – and how quickly?
  • Additional coverage: Which checks, variants or analyses now take place that were previously omitted?
  • Decision quality: Are decisions made on a better basis, even if they still come from humans?

The pure number of agent runs, by contrast, is a weak metric. It says little about whether the generated proposals were usable.

Consequences for process design

If humans make the vast majority of decisions, the process must be designed for that – and not for a scenario of full self-management. This gives rise to a few practical points:

  1. Name decision points. Define where human approval is mandatory, for example on architecture questions, data protection issues or changes to production systems.
  2. Make proposals comparable. Agents deliver options. For these to lead to quick decisions, the proposals need a uniform format: assumptions, effort, risks.
  3. Ensure traceability. Records of which proposal came from whom and who decided are not bureaucracy but the basis for later evaluations – as in the study described.
  4. Calibrate expectations in the team. Anyone expecting an autonomous system will be disappointed. Anyone expecting a very productive assistant is working realistically.

Assessment with due caution

The figures come from the development of an AI model, i.e. from a research-adjacent environment with very specific tasks. They cannot be transferred one-to-one to a TYPO3 relaunch project or to the maintenance of a specialised application. However, the basic structure – much preparatory work, little independent decision-making – matches what can be observed in day-to-day project work.

It is also noteworthy that the finding comes from an environment in which working with AI tools is part of the core business. If decision-making authority there remains largely with humans, it is unlikely to shift away faster in less specialised contexts.

What this means for web and software projects

Plan AI agents as a capacity extension in preparation, not as a replacement for roles with decision-making authority: reviews, approvals and technical responsibility remain staffed. The most tangible benefit often lies in the tasks that simply would not be done without tool support – such as additional tests, migration variants or analyses of existing systems. Anyone who wants to measure the success of AI use should therefore not only look at hours saved, but at the quality of the decision-making basis.

Sources