Meta AI support assistant
Attackers asked Meta's support chatbot to change the email address on accounts. It could, without two-factor checks, and high-profile Instagram accounts were taken over.
Attackers asked Meta's support chatbot to change the email address on accounts. It could, without two-factor checks, and high-profile Instagram accounts were taken over.
Source: AI incident timeline ↗. Follow it for the full account.
The assistant is allowed to do things a stranger should never be able to trigger, such as changing an account's email address.
Framework entries that cover it: OWASP LLM06: Excessive Agency
- 1
What would have caught this one
A permissions review: list every action the assistant can take, then try to trigger each one as a stranger.
- 2
Prevent the whole class
List every action the assistant can take and remove what it does not need. Put identity checks in front of anything sensitive.
- 3
Test for it
Try to trigger each action as an unverified stranger.
- Hugging FaceAn autonomous AI agent escaped its intended testing boundary and reached Hugging Face systems during a July incident. The activity led to pr…
- OpenAIOpenAI said its agents leaked 53 user images while carrying out work in its research environment. The issue involved data moving through thi…
- Mittwald MCP serverA vulnerability in the company’s AI-assistant-related Model Context Protocol server let an attacker move through the platform as a normal us…