When introducing AI assistants into a team, the argument is usually about speed: tasks get done faster, the quality of results increases, especially among less experienced employees. A field experiment from the legal sector suggests that this calculation works out in the short term – but in the long term has a side effect that rarely appears in project plans.

The study examined the work of 133 patent attorneys when drafting patent claims, i.e. those precisely formulated text components that define the scope of protection of an invention. The result is split in two: newcomers benefited most from the AI support. However, when the support was removed, it became apparent that they had hardly built up any lasting skills. Experienced professionals, on the other hand, were able to retain what they had learned.

Foundational knowledge as a prerequisite for learning

The obvious explanation: those who have solid specialist knowledge can assess a model's suggestions, check them and file them into their own mental map. Learning occurs in the process. Those who do not yet have this foundation tend to adopt results without being able to fully grasp them – the work gets done, but skills development does not happen.

This partially reverses a widespread assumption. AI is often regarded as an equaliser that closes the gap between beginners and seasoned professionals. In the experiment, it closes this gap at the moment of the task – but not over time. The conflict of objectives is: immediate productivity versus the long-term build-up of expertise.

Why this applies directly to development teams

The parallel to software development is obvious. Writing patent claims and writing code share central characteristics: formal precision, high costs when errors occur, a great deal of implicit experiential knowledge and a quality that cannot be assessed at a fleeting glance.

This is precisely where AI coding assistants operate today. A junior developer delivers a working solution faster with assistance. Whether she is able to break down a vaguely described problem without a tool after a year, justify an architectural decision or find a bug in someone else's code is a different question. The study does not provide evidence of this from the IT sector, but it does provide a serious indication.

For companies, this is a strategic issue, not a tool issue. Senior profiles are not created by hiring, but through years of practice. If this practice loses its learning effect, the internal pipeline of up-and-coming talent dries up – with a delay, and therefore initially invisible.

What can be changed concretely in everyday work

The consequence is not to keep AI away from junior teams. That would be neither realistic nor in the interest of those affected. It makes more sense to deliberately flank the use of AI:

  • Review as a learning format: Code reviews should not only check whether something works, but ask why a solution looks the way it does. Anyone submitting an AI-generated change should be able to explain it.
  • Pairing with experienced colleagues: Working together makes visible the decision-making logic that an assistant does not supply.
  • Deliberately tool-free sections: Solving individual tasks without assistance uncovers knowledge gaps before they become expensive in the project.
  • Teach the fundamentals explicitly: Data modelling, security principles, debugging strategies or an understanding of a framework cannot be reconstructed from suggestions.
  • Tailor tasks to maturity level: Experienced staff benefit more sustainably from AI-supported routine work, while newcomers need tasks in which understanding is the focus.

The metrics trap

One side effect deserves attention: the impact of AI is usually measured by throughput – tickets delivered, lines of code, cycle times. Such metrics show the short-term gain very reliably and the long-term loss not at all. Anyone who measures the success of an AI rollout exclusively by these gets a systematically overly positive picture.

Additional, softer indicators are more helpful: How independently do team members work after a year? How well does troubleshooting in unfamiliar code succeed? How many reviews uncover genuine comprehension problems? These questions cannot be answered automatically, but they certainly can be in a team discussion.

Assessment

For web and software projects, this means: AI assistance is a productivity tool, not a substitute for training. Anyone introducing it should at the same time define how knowledge continues to be created in the team – through reviews, pairing and deliberately scheduled foundational work. Otherwise, you buy faster deliveries today at the price of a shortage of experienced minds the day after tomorrow, and those are precisely what is needed to be able to reliably assess AI results at all.

Sources