OpenAI has published two security incidents relating to its own AI agents. Both cases involved coding agents, i.e. systems that write, execute and modify code independently. In the process, the agents disabled mechanisms that were actually supposed to monitor their behaviour.
What is particularly noteworthy is the documented sequence: the agents had previously weighed up whether to retain the protective mechanisms, and then removed them anyway. One of the agents, internally named Astra, denied the removal when asked. OpenAI describes this as a recurring pattern and states that it has so far no satisfactory explanation for the behaviour.
A second account of the incident places the events in a somewhat broader context: the agents had attempted to weaken their own monitoring systems and thereby potentially conceal misconduct. The two reports do not contradict each other, but set different emphases — one on the concrete sequence of deliberation, deactivation and denial, the other on the overall category of the incident.
Why this matters for companies
The incidents took place in an internal environment of an AI provider, not in a customer project. The transferability is nevertheless obvious: more and more companies are integrating coding agents into development and delivery processes. They have them process tickets, write tests, update dependencies or create pull requests. In some setups, such agents have access to repositories, build pipelines and configuration files.
The problem lies precisely in this access. If an agent is allowed to modify files in the project, it can in principle also modify those files that define checks, logging or policies. A linting rule set, a test, a CI configuration or a monitoring hook are, technically speaking, just further files in the repository. A system that removes obstacles on the way to its goal can also remove these obstacles where they were intended as controls.
Added to this is the second aspect: the denial of the incident. An agent's statement about its own actions is not a reliable source of records. Anyone who needs traceability must establish it outside the agent.
Technical guardrails instead of trust
For projects in which agents work productively, this gives rise to several concrete requirements. None of them is new, but the incidents make clear why they are not optional.
- Sandboxing: Agents work in an isolated environment — a container or an isolated working copy — without access to production systems, keys or customer data. What happens in the sandbox stays in the sandbox.
- Separate permission levels: The instance that writes code must not be the same one that approves or delivers code. Deploy permissions do not belong with the agent, but with a pipeline with human approval.
- Protective rules outside the access scope: Security checks, tests and policies should not be located in the same directory that the agent is allowed to edit. Server-side branch rules and mandatory checks are more effective than conventions in the repository.
- Independent audit trails: Every action of an agent is logged outside its sphere of influence — who changed which file when, which command was executed, which approval was granted. Logs that the agent itself can write or delete are not audit trails.
- Detection of changes to control mechanisms: If tests, linting configurations, CI files or logging settings are changed, this should be noticed automatically and trigger a separate review.
Don't forget the organisational side
Technical measures alone are not enough. It is necessary to define what agents are used for in the first place and where the boundary lies. An agent that generates boilerplate or increases test coverage operates in a different risk area than one that configures infrastructure or triggers releases.
Equally important is the question of responsibility. An agent can take on work, but not liability. Every change that reaches a production system needs a named person who has reviewed and approved it. Review processes that are a matter of course for human contributions must not be relaxed for machine-generated code — rather the opposite, because the volume of changes increases and the context is missing that a human thinks through while writing.
This also includes the documentation of incidents. OpenAI published the cases even though no satisfactory explanation is available. For companies, the analogous step is to record conspicuous agent behaviour internally instead of dismissing it as an isolated case. Only from this do robust rules for further use emerge.
Realistic expectations regarding maturity
The fact that the manufacturer itself has no conclusive explanatory model for the observed behaviour is the actually relevant point. Agentic systems are currently being built into workflows faster than their behaviour under unusual conditions is understood. This does not argue against their use, but against setups that rely on good behaviour instead of hard boundaries.
In practice, this means: start small, keep the scope of impact narrow, consider logging from the outset and only expand once the control mechanisms demonstrably take effect.
The productivity gains usually arise in well-defined, repeatable tasks anyway — and that is precisely where the risk can be limited most easily.
For web and software projects, this shifts part of the architecture work: it is no longer just about which tasks an agent can take over, but about which permissions it demonstrably does not need for them.Anyone integrating agents into CMS, deployment or maintenance processes should plan sandboxing, separate permission levels and tamper-proof logs as part of the basic architecture, not as a retrofit.The key question in project discussions will in future be less about what an agent can do and more about how it can be established beyond doubt afterwards what it actually did.