How much work do AI agents actually take over in software development? Until now, the answer has mostly been based on anecdotes. OpenAI has now published figures on its own use for the first time – and they are revealing in both directions: they show very intensive use, but also state just as clearly where the tools reach their limits.
What the figures say
According to the analysis, the median researcher consumed tokens in October with a theoretical equivalent value of 2,400 US dollars per day. Tokens are the smallest processing units of language models; consumption is usually billed on that basis. The figure is to be understood as a calculation – internally, the list prices do not apply in this way – but it shows the scale: agents are not running there occasionally, but continuously.
Even more telling than the consumption is the assessment of those involved themselves. The researchers stated that they would rather do without 40 percent of their colleagues than without their coding agents. That is a subjective preference and not a productivity measurement – but a remarkable one, because it comes from an environment that has very good human specialists at its disposal.
Where the agents fail
The same analysis names the other side without glossing over it. With complex infrastructure and with new ideas, the agents fail regularly. And the most important concepts for the coming model generations GPT-6 and GPT-6.2 came exclusively from humans.
This paints a fairly clear picture: agents are strong where a problem can be described, verified and solved using known patterns. They are weak where a system has to be understood in its entirety, where implicit knowledge about grown dependencies is needed, or where a solution path first has to be invented.
Transferability to everyday agency work
OpenAI is not a typical mid-sized company and not a web agency either. The framework conditions differ in several points that should be taken into account:
- Cost framework: Token consumption of this magnitude per head and day is simply not budgetable in most organisations. Anyone who uses agents productively works with much tighter limits – and must therefore select more precisely which tasks are worthwhile.
- Task profile: Research-related work on models produces many well-isolatable experiments. Project business with customer systems, legacy code and interfaces to third-party systems looks different – and thus tends to fall into the category in which, according to OpenAI, the agents are weak.
- Responsibility: Anyone who delivers software to customers bears liability for the result. An agent suggestion is a draft, not an accepted state.
Which tasks can sensibly be delegated today
A pragmatic division can be derived from the pattern described. Tasks with a clear description of objectives and quick verifiability are generally well suited:
- Writing tests and extending existing test coverage
- Routine refactorings within clearly delimited modules
- Migrations according to a known schema, such as adjustments to new framework versions
- Initial drafts for documentation, changelogs and code comments
- Error search in reproducible, narrowly scoped cases
- Recurring boilerplate work, i.e. structurally uniform standard code
More restraint is advisable where architectural decisions are made, where security-relevant logic is created, where permission and data protection concepts are touched, or where a system is changed across several levels. These are precisely the areas that appear as a weak point in the OpenAI analysis – and there the verification effort is often higher than the time gained.
What teams need organisationally
The benefit of agents depends less on the model than on the environment in which it works. The better a project can be verified automatically, the more can be delegated. In practice, this means: robust tests, a functioning continuous integration pipeline, i.e. automated build and check runs on every change, as well as clean code reviews with human approval.
Equally important is sober cost control. If an agent tries out variants over hours, costs arise that only pay off if the result is actually used. It is worth monitoring consumption and benefit per type of task instead of switching agents on across the board everywhere.
And finally: specialist conceptual work remains human work for the time being. The fact that even at OpenAI the central ideas for the next model generations came from humans is a clear signal against the expectation that agents could take over the conceptual phase of a project. Requirements clarification, architecture design and the weighing of specialist objectives against technical feasibility remain with the team.
Classification for web and software projects
For web and software projects, this means: coding agents are a serious tool for well-describable routine work, but not a replacement for conception, architecture and responsibility. They have the greatest leverage in projects that are already well structured, tested and automatically verified – anyone who invests here simultaneously improves the starting position for the use of agents.
Realistically calculated, the gain is therefore not a blanket acceleration, but rather relief on clearly defined tasks, while the complex decisions continue to be made by the team.