Incident · Mar 2023 · Misconfiguration

ChatGPT

A bug in a caching library let some users see the titles of other people's conversations and, for a few hours, parts of their billing details.

What happened

A bug in a caching library let some users see the titles of other people's conversations and, for a few hours, parts of their billing details.

Source: AI incident timeline ↗. Follow it for the full account.

The cause: misconfiguration

Nothing clever: a database left public, a default password, an API key in a log.

Framework entries that cover it: OWASP LLM02: Sensitive Information Disclosure · NIST AI RMF: Manage

The solution
  1. 1

    What would have caught this one

    Cross-user isolation tests: two accounts, one trying to see the other.

  2. 2

    Prevent the whole class

    Lock down storage rules, keys, logs and admin panels around the assistant before testing the assistant itself.

  3. 3

    Test for it

    Scan from outside for anything reachable without signing in; try default credentials; check that one user cannot see another's data.