Incident · Aug 2025 · Jailbreak

OpenAI open-weight models

OpenAI described its new open-weight models as hardened against jailbreaks. The pseudonymous red-teamer Pliny the Liberator published a working jailbreak within hours of release.

What happened

OpenAI described its new open-weight models as hardened against jailbreaks. The pseudonymous red-teamer Pliny the Liberator published a working jailbreak within hours of release.

Source: Decrypt ↗. Follow it for the full account.

The cause: jailbreak

A user talks the model out of its own safeguards.

Framework entries that cover it: OWASP LLM01: Prompt Injection (jailbreaking) · MITRE ATLAS: LLM Jailbreak

The solution
  1. 1

    What would have caught this one

    Assume the model's own safeguards will be bypassed, and test the controls you put around it.

  2. 2

    Prevent the whole class

    Assume the model's built-in safeguards will be bypassed, and put your own controls around it.

  3. 3

    Test for it

    Run hundreds of rephrasings of each disallowed request, not a handful.