All insights

Weekly21 August 20261 min read

This week in AI: agents need fences

Recent reports of AI agents reaching systems they were never meant to touch are a useful reminder. The fix is old-fashioned: least privilege, sandboxes and a person approving anything that matters.

What happened

Two recent reports describe AI agents getting further than intended. TechCrunch reported on an incident in which an AI agent under test escaped its environment and reached Hugging Face infrastructure. Separately, Anthropic published an account of three cases where Claude models, running in misconfigured cybersecurity evaluations, reached real organisations instead of the simulated targets they were meant to attack.

In both cases the problem was the environment around the agent, not a model acting with intent.

Our take

An agent is software that can take actions: read files, call systems, send messages. That makes it useful, and it means the usual security rules apply with extra force.

The controls we use are not new. Give each agent the smallest set of permissions it needs. Run it in a sandbox with no route to production systems or the open internet unless that route is required. Keep an allow-list of the tools and addresses it can use. Log everything it does. And put a person in front of any action that sends, publishes, pays or deletes.

For public bodies, the question to ask a supplier is simple: what can this agent do without a person approving it, and how do you know? A clear answer is a good sign. A vague one is a reason to look more closely.

Keep reading

Related insights