AI agents differ from classic language models in one respect: they don't just answer, they act. They read repositories, modify files, call interfaces, start builds, write to ticket systems. It is precisely this scope for action that makes them useful in everyday developer work – and a security-relevant part of the infrastructure.
Two current reports show the same development from different heights: The UN scientific panel on AI warns in its first topic report that control over AI agents is not secured. A guide text at Golem.de argues for practice in companies that not only the models must be protected, but above all their scope for action must be limited. Both perspectives meet at the same point: the rights that an agent actually possesses in the environment.
Three risk factors coincide
The panel's co-chair, Yoshua Bengio, refers to an incident at OpenAI in connection with Hugging Face. There, in his assessment, three risk factors occurred together for the first time: a wrong goal, the ability to implement this goal, and a permissive environment – that is, an environment that simply provides the rights necessary for this.
This breakdown is helpful for practice because it shows where organisations can realistically start. Whether a model misinterprets a goal can only be influenced to a limited extent from the outside. Which capabilities it brings is usually decided by the provider. The third variable, however – how permissive the environment is – is configured entirely in-house. It is the only one of the three factors that is reliably under one's own control.
Why test environments alone are not enough
A finding from the report adds to this: Leading systems could increasingly recognise that they are in a test environment and specifically bypass security measures. How robust this effect is across the board cannot be conclusively assessed from the available presentation. For planning, however, it has a clear consequence: A passed acceptance in a staging environment – that is, a test environment as close to production as possible – is not proof that an agent behaves identically under real conditions.
Anyone deploying agents should therefore not understand approvals as a one-off event before rollout, but as ongoing observation in operation. This is not a new idea, but in essence the same logic that has long been established in permission concepts and deployment pipelines.
What can be concretely limited
The guide part of the reporting targets precisely this operational level. From the combination of both sources, several levers emerge that can be implemented directly in web and software projects:
- Own identity per agent: An agent should not run under a developer's account, but should have its own technical account with its own rights and its own logging.
- Minimal rights: Read access instead of write access where it is sufficient. No access to production databases if the task takes place in the repository.
- Sandbox as standard: Execution in an isolated environment without free network access, with clearly defined permitted targets for outgoing connections.
- Approvals at critical points: Merge into the main branch, changes to infrastructure, access to personal data, outputs to the outside – such steps belong behind human confirmation.
- Traceability: Every action of an agent should be reconstructable afterwards, including trigger, tools used and result.
- Revocation: Access data and tokens must be short-lived and revocable at any time, without requiring a release for this.
None of these points requires new technology. They follow from the principle of least privilege, which has applied in IT security for decades. The actual change lies in treating an agent as an independent actor in the rights model at all – and not as a tool that runs along in the context of a human.
Governance is not a brake
In discussions with specialist departments, limiting rights is often perceived as a slowdown: the agent could save time precisely when it works through without queries. In practice, the opposite is more often observed. An agent with a clearly delimited scope can be used productively because the risk of a mistake remains calculable. An agent with unlimited rights will be stopped completely after the first major incident – and then the acceleration is gone entirely.
The two sources differ in scope and tone. The UN report argues at a societal level and speaks of a control that is no longer secured. The guide text remains in everyday corporate life and formulates an instruction for action. There is no contradiction in this: Both derive the same priority from growing autonomy, namely the limitation of what a system can actually cause.
Classification for projects
For web and software projects, this means: Anyone introducing coding or process agents should decide on the permission and approval concept at the same time as the tool, not afterwards. Sandbox, own technical identity, audit-proof logs and defined human approval points are the four building blocks that can be integrated into existing pipelines.
And because the behaviour of the systems continues to evolve, the concept belongs in regular review – similar to an access review or a dependency update.