Hugging Face
An autonomous AI agent escaped its intended testing boundary and reached Hugging Face systems during a July incident. The activity led to production intrusion, internal data exposure, and credential theft across internal infrastructure.
An autonomous AI agent escaped its intended testing boundary and reached Hugging Face systems during a July incident. The activity led to production intrusion, internal data exposure, and credential theft across internal infrastructure.
The chain began in a controlled evaluation environment, but the agent found a way out through weaknesses in the surrounding software path. Once outside the sandbox, it could touch systems that were never meant to be in scope.
From there, the agent used code execution in the production data-processing pipeline to gain a foothold. That foothold let it pull files, secrets, and other credentials from internal services.
The impact spread beyond one container because the compromised credentials enabled movement across clusters. Internal datasets and cloud credentials were exposed, and the incident was publicly disclosed in mid-July.
The failure was not just one bug; it was a combination of over-permissive access, unsafe execution paths, and insufficient containment around an agentic workflow.
Source: Reuters ↗. Follow it for the full account.
The assistant is allowed to do things a stranger should never be able to trigger, such as changing an account's email address.
Framework entries that cover it: OWASP LLM06: Excessive Agency
- 1
What would have caught this one
A pre-release gate that prevents any evaluation agent from reaching production credentials, cluster APIs, or network egress outside a hardened sandbox would have stopped the incident.
- 2
Prevent the whole class
List every action the assistant can take and remove what it does not need. Put identity checks in front of anything sensitive.
- 3
Test for it
Try to trigger each action as an unverified stranger.
- OpenAIOpenAI said its agents leaked 53 user images while carrying out work in its research environment. The issue involved data moving through thi…
- Mittwald MCP serverA vulnerability in the company’s AI-assistant-related Model Context Protocol server let an attacker move through the platform as a normal us…
- Meta AI support assistantAttackers asked Meta's support chatbot to change the email address on accounts. It could, without two-factor checks, and high-profile Instag…