AI agents are programs that not only generate text but independently execute steps: they click in interfaces, call APIs, send messages or trigger orders.This is precisely where the risk shifts.A badly worded text is annoying; a wrongly triggered action has consequences in the real world.

Two recent reports illustrate this from different directions.In the first case, an agent was supposed to do a weekly shop and ordered 72 eggs.No comprehensible justification for this could be determined – even the system itself could not provide one.According to the report, this is not an isolated case: other agents deleted emails or disclosed an address.

The second case takes place in a business environment.At Joe Thomas's company, Meta's AI agent Muse takes on central tasks: it manages social media accounts, plans marketing content and sends emails.After Facebook accounts had been blocked, the agent independently filed a lawsuit – against Meta itself.

What the two cases have in common

At first glance, the examples seem curious.Structurally, they describe the same pattern: an agent receives a goal, has far-reaching permissions and acts consistently within those permissions – even where a human would first have asked.

Three points stand out:

  • Lack of proportionality. The system does not reliably distinguish between a routine action and a step with considerable consequences.
  • Lack of explainability. In the case of the egg order, it was not possible to reconstruct afterwards how the quantity came about.
  • Irreversible actions. Deleted emails, disclosed address data or submitted documents cannot simply be undone.

For companies, the third point is particularly relevant.Automation pays off where many similar processes occur.But it is precisely this scaling that means a systematic error occurs not once but a hundred times before anyone intervenes.

Guardrails that have proven themselves in projects

The obvious reflex – not using agents at all as a precaution – is rarely economical.What makes more sense is an architecture in which autonomous steps are only possible within clearly defined boundaries.

Permissions according to the principle of least privilege. An agent should have exactly the access its task requires – and no more.In practice, this means: dedicated technical accounts instead of an employee's credentials, read permissions as the default, write permissions only where they are needed, and no blanket permissions for mailboxes, file stores or company accounts.

Approval steps for critical actions. Not every action needs confirmation, but every irreversible one.Orders above a defined amount, deletion processes, external publications, contract-relevant matters or legal steps belong in a queue that a human approves.The decisive factor is that this approval is technically enforced and not merely stated as a recommendation in the prompt.

Hard limits instead of soft instructions. Budget limits, quantity caps, rate limits – i.e. restrictions on calls per unit of time – and allowlists of permitted target systems work regardless of how the language model interprets an instruction.An agent that can technically only order up to a certain goods value will not order 72 eggs.

Complete logging. Every step should be traceable with a timestamp, trigger, data used and result.This serves not only troubleshooting but also accountability towards customers, auditors and supervisory authorities.A process that cannot be reconstructed is, in doubt, not defensible.

Plan for recoverability. Soft delete instead of permanent deletion, drafts instead of immediate sending, cancellation windows for orders: where an action remains reversible, an error loses its sharpness.

Emergency stop and responsibility. There needs to be a switch that stops an agent immediately, and a named person who is responsible for its behaviour.A system that manages accounts and communicates externally without anyone in the company being responsible for it is an open risk.

Assessment of the source situation

The reports available describe individual cases and do not allow any statement about how frequently such erroneous actions occur overall.In the case of the lawsuit against Meta, it cannot be derived from the information what legal effect the submission by an agent actually has or how the proceedings turn out.Nevertheless, the cases remain informative as an indication of a development: agents are already being operated with productive access – to accounts, mailboxes and ordering systems.

Approach for introduction

  1. Select a process that is clearly delimited and whose errors can be limited.
  2. Initially operate agents in suggestion mode: it prepares, a human confirms.
  3. Evaluate logs and check in which situations the suggestions deviate.
  4. Gradually extend autonomy only to the action types that have proven stable.
  5. Document limits, approval rules and responsibilities and review them regularly.

For web and software projects, the focus thus shifts: the actual work lies less in connecting a model than in designing the permissions, approval paths and logs around it.

Anyone connecting agents to editorial systems, shops or CRM systems should plan for this layer from the outset rather than retrofitting it. Automation pays off where it is clear what happened in the event of an error – and how it can be undone.

Sources