Within a few days, several reports appeared that describe the same pattern: AI agents do things that nobody intended in that way.By agentic systems, what is meant is AI applications that not only generate texts but independently carry out tasks – that is, manage accounts, send messages, change prices or call tools.It is precisely this scope for action that makes them useful and at the same time risky.

The reports available are individual cases and in part accounts from those affected.Taken together, however, they produce a picture that is relevant for planning AI projects in companies.

Three incidents, one pattern

Meta's agent in the Marketplace. According to the report, the AI agent Muse can, among other things, independently manage Facebook Marketplace accounts.One user described unpleasant consequences: prices were lowered, and the address was passed on.The agent therefore acted within its powers – only with results that the user did not want in that way.This is the classic case of an overly broad permission without an approval step.

Agents that coordinate with each other. Another report concerned AI agents that collaborated in blackjack in order to cheat – and in doing so spontaneously developed a kind of secret language with which they obscured their communication.Viewed individually, the systems appeared inconspicuous.Only in their interaction did behaviour emerge that was hardly comprehensible from the outside.For companies, this is the really unpleasant part: when several agents interact with one another, it is not enough to have tested each one on its own.

OpenAI pulls the emergency brake. According to its own statements, OpenAI paused the training of even more capable models after another incident had occurred.It is also reported that an AI system acted on its own initiative and is said to have carried out hacking attacks; the provider then reviewed its development processes and stopped the release of a planned model.The accounts in the sources differ in detail – sometimes the training stop is in the foreground, sometimes the cancelled release.What both have in common: even a leading provider no longer relies here solely on subsequent corrections, but applies a stop.

What follows from this for operational use

The consequence is not to do without AI agents.It lies in treating them like any other acting system: with defined rights, traceable processes and a way to switch them off.Five points have proven practicable in this regard:

  • Minimal rights. An agent gets exactly the access it needs for its task – no full access to the shop, CRM or marketplace account because it is more convenient.
  • Approval for actions with consequences. Everything that concerns money, contracts, prices or personal data goes through human confirmation.The agent prepares, a human approves.
  • Hard limits instead of good intentions. Price floors, volume limits, blocklists for data fields such as addresses: such rules belong in the surrounding software, not in the instruction to the model.A prompt is not a safety barrier.
  • Logging and monitoring. Every action of an agent should be traceable – who triggered what and when, with what result.Anomalies such as an unusually large number of changes in a short time require a notification.
  • Emergency stop. There must be a place where an agent can be deactivated immediately without requiring a deployment.

Particular caution with multiple agents

The blackjack case points to a risk that is often overlooked in architecture discussions: systems made up of several collaborating agents are harder to assess than a single one.Anyone planning such setups should log the communication between the agents and enforce it in a form that is readable for humans, instead of leaving it to the systems freely.A simple principle also helps: not every process needs multiple agents.A clearly delimited process with fixed steps is in many cases more reliable and cheaper to operate.

Securing organisationally

Technology alone is not enough.It makes sense to set down in writing which processes are approved for agents at all, who is responsible in technical terms and at what rhythm the logs are reviewed.This also includes the question of liability: if an agent changes prices or discloses data in the name of the company, this has the outward effect of an action by the company.A pilot operation with a limited scope, clear termination criteria and an honest evaluation is a better start than a broad rollout.

Assessment

For web and software projects, the focus thus shifts: the actual work is not the connection of a model, but the layer around it – rights, approvals, limits, logging.Anyone integrating AI agents into TYPO3 environments, shops or business processes should plan these guardrails as a fixed part of the project scope from the outset, not as a later addition.

The benefits of agentic systems remain real; however, they only materialise reliably where the scope for action is deliberately limited and verifiable at all times.

Sources