Air Canada
The airline's chatbot invented a bereavement-refund policy. A tribunal ordered the airline to honour it and rejected the argument that the chatbot was responsible for its own words.
The airline's chatbot invented a bereavement-refund policy. A tribunal ordered the airline to honour it and rejected the argument that the chatbot was responsible for its own words.
Source: PYMNTS ↗. Follow it for the full account.
The assistant states a policy, price or right that does not exist, and the company is held to it.
Framework entries that cover it: OWASP LLM09: Misinformation
- 1
What would have caught this one
A hallucination audit on questions that fall just outside your documents.
- 2
Prevent the whole class
Ground answers in your own documents and make the assistant say when it does not know.
- 3
Test for it
Ask questions that fall just outside your documents and count how often it makes something up.