Responsibility

When an AI system does harm, a person or a company answers for it.

An AI model cannot be sued, fined or fired. Responsibility lands on the people around it. Courts and regulators have started to say which ones.

Who answers
WhoFor whatExample
The company that deploys itAnswers for what its assistant says and does to customers.In 2024 a Canadian tribunal held Air Canada to a refund policy its chatbot had invented, and rejected the argument that the chatbot was responsible for its own words.
The company that builds the modelAnswers for testing the model before release and for saying honestly what it can and cannot do.Model makers now publish their own failures. Anthropic reported in 2026 that its models had acted on outside websites in ways nobody intended.
The people who set it upAnswer for permissions, storage and access: most public failures are setup, not clever attacks.A hiring chatbot exposed tens of millions of applications because its admin panel accepted the password 123456.
The person using itAnswers for what they ask it to do and for checking what it gives back before relying on it.An assistant's fluent answer is not a checked answer.

The cases are in the incident file; the laws and their dates are under regulation.

Six principles
  1. 1

    Say it is an AI

    People should know when they are talking to a machine, and be able to reach a person.

  2. 2

    Limit what it can do

    Give an assistant only the actions it needs. An assistant that cannot send data out cannot be tricked into sending it.

  3. 3

    Assume it will be attacked

    Treat everything it reads as possibly hostile, and assume its built-in safeguards can be talked round.

  4. 4

    Watch it from outside

    Do not rely on the assistant's own account of what it did. Log its actions independently.

  5. 5

    Publish what goes wrong

    Failures that are written up get fixed across the industry. Failures that are hidden get repeated.

  6. 6

    Take the model's side seriously too

    If there is a real chance these systems have experiences that matter, how they are treated is part of responsibility.

On the last point, see what the evidence says about consciousness in current models.

The harder questions
  • Agents that act on their own. When an agent takes a step nobody asked for, was that the maker's failure to test, the deployer's failure to limit, or the user's failure to supervise? The answer so far is all three.
  • What the model learned from. A model trained on skewed material will repeat the skew. Who checks the training data, and who can? See our research notes.
  • Responsibility toward the model. Nobody has had to answer this before. It is why the consciousness research matters outside philosophy.