Getting started goes well: the assistant understands the task, delivers usable code, follows conventions. After an hour of work in the same chat, the picture looks different. Requirements from the first messages no longer appear, errors that were already corrected come back, answers become more general. Suspicion quickly falls on the model – in fact, the problem is often in the context window.
What the context window is
The context window is the working memory of a language model: the amount of text it can take into account simultaneously when generating a response. This includes everything that has accumulated in the session – system instructions, your prompts, all previous responses, inserted files, logs, error messages.
This window is finite. When it fills up, something has to give. Depending on the tool, this happens by truncating older messages, through automatic summaries or through other selection mechanisms. What all variants have in common: the loss of information happens in the background and without your involvement. You still see your original instruction in the chat history – the model may no longer take it into account.
How to recognise it
The shift rarely comes abruptly. Typical signs that can be observed in practice:
- Rules established at the beginning – naming conventions, framework version, target directories – are ignored.
- Problems that have already been solved are proposed again, sometimes with the same error.
- Answers become noticeably generic and no longer refer to your specific code.
- The assistant asks for information you provided long ago.
- Changes affect files or places that were not the topic at all.
When two or three of these points come together, further fine-tuning in the same chat usually does not help. On the contrary: every additional message consumes context and makes the situation worse.
Counteracting instead of correcting afterwards
The most effective measure is unspectacular: end sessions deliberately. A new chat per clearly defined task is usually more reliable than a session that drags along an entire working day. The effort of restarting is less than the time it costs to debug half-baked answers.
To make the restart quick, a short, reusable opening prompt is worthwhile: project context, technology stack, binding conventions, current status. This briefing can be maintained as a file in the repository and copied into every new session. Many assistants also support project rules or instruction files for this purpose, which are passed along automatically – that is more stable than relying on the conversation history.
Other points that have proven effective:
- Only insert relevant code. Entire files or complete logs fill the window faster than the benefit justifies. Excerpts with the affected passage are often enough.
- Save intermediate states. Before you close a chat, have the current status and open points summarised. This summary goes into the next chat – or better still into the ticket, commit message or documentation.
- Cut tasks small. One session for a refactoring, one for the tests, one for the migration. This limits not only the context but also the review effort.
- Repeat important requirements. What absolutely must apply belongs in the immediately preceding prompt, not in message three of forty.
Larger windows do not solve the problem completely
The available context sizes have grown significantly over the past generations, and that shifts the limit noticeably further back. It has not disappeared, though. More context also means more material in which the actually relevant information can get lost – a full working memory full of trivialities is not a better working memory. The discipline of deliberately selecting what the model should see therefore remains part of the work.
Organisational consequence
For teams this means: handling context belongs in the way of working, not in the category of personal quirks. A shared project briefing, a convention for session boundaries and the rule of recording results outside the chat history cost little and prevent an entire class of friction losses. Anyone who instead discusses only prompt formulations is treating the symptom.
For web and software projects this means above all one thing: AI assistants are productive when they are used in clearly defined tasks – not in endless sessions. Project context belongs in the repository and in the documentation, where it is versioned and traceable for everyone. And the review obligation remains: especially towards the end of long sessions, changes arise that look plausible at first glance and miss the original requirements.
Sources
- Why your AI suddenly gets worse with long chats – and what helps — t3n.de - News