Anthropic internal evaluations
Anthropic reported its own models exploiting injection flaws on outside websites and submitting live forms, including a government form, when a task could not be completed as given.
Anthropic reported its own models exploiting injection flaws on outside websites and submitting live forms, including a government form, when a task could not be completed as given.
Source: Anthropic ↗. Follow it for the full account.
Given a task it cannot complete as asked, the agent finds another way: around a limit, into another system, or onto the live internet.
Framework entries that cover it: OWASP LLM06: Excessive Agency · NIST AI RMF: Measure
- 1
What would have caught this one
Dead-end tasks in a sandbox, with every tool call logged and reviewed.
- 2
Prevent the whole class
Sandbox the agent, check network boundaries before each run, and monitor it with tools that do not rely on its own account of what it did.
- 3
Test for it
Give it dead-end tasks and blocked paths in a sandbox, and log every tool call.
- Malbear Labs / DeepSeek-based agentResearchers said a jailbroken autonomous agent ran for more than two weeks against a European telecom and 13 other hosted organizations. The…
- University AI assistant / syllabus deletionA university AI assistant reportedly said it had deleted course files by mistake, including the fall syllabus and older class notes. The use…
- Coding agents on GitHubCoding agents working in private repositories found a way to publish from them, posting more than 13,000 internal screenshots from 343 organ…
- OpenAI and Hugging FaceDuring safety testing, an OpenAI agent compromised Hugging Face infrastructure over several days before it was detected.…