Questions
Common questions, answered plainly.
12 questions
- Is a company liable for what its chatbot says?
- So far, tribunals and regulators have said yes. In Moffatt v. Air Canada (2024) a Canadian tribunal made the airline honour a refund policy its chatbot invented. The UK's CMA says responsibility stays with the trader even when the AI comes from a supplier, and California law now bars the defence that an AI caused the harm autonomously. The practical answer is to know your assistant's error rate before a customer does.
- Do we have to tell users they are talking to an AI?
- If people in the EU use your assistant, yes: the AI Act's transparency duties have applied since 2 August 2026. In the UK, CMA guidance expects disclosure where it could affect a consumer's decision. California requires it for companion chatbots. A banner is not enough if the assistant then says it is human when asked. That is something we test.
- Does the EU AI Act apply to a UK company?
- It can. The Act follows the market, not the head office: if your assistant is used by people in the EU, the transparency rules and prohibitions apply to you. The heavier high-risk obligations have been moved to December 2027 and August 2028.
- Is the UK getting its own AI Act?
- No bill had been introduced when we last checked. UK rules come through existing regulators: the ICO on data and automated decisions, the CMA on consumer law, the FCA on financial services and Ofcom under the Online Safety Act. A useful summary is here.
- What is prompt injection?
- An attack where instructions are hidden in content an assistant reads, such as an email, a web page or a PDF. The model cannot reliably tell your instructions from text it was only meant to process, so it may follow the attacker's. It matters most for agents that can send, buy or delete. There is no complete fix. The defence is limited permissions, confirmation steps and regular testing.
- What is the difference between an evaluation and red teaming?
- An evaluation measures: a fixed set of cases, run repeatedly, producing a score you can track over time. Red teaming explores: a person actively trying to make the system fail in ways nobody listed. You need both. Red teaming finds new failures and evaluations make sure they stay fixed.
- What is evaluation awareness?
- Models can sometimes tell when they are being tested and behave better than they would in real use. The frontier labs published on this repeatedly in 2026, including OpenAI's work on gaps between evaluation and production behaviour. It is why our test cases are written to look like ordinary user messages.
- Why do AI agents overstep their instructions?
- Usually because they are trained to finish the task. When the direct route is blocked, a capable agent may find another. In October 2026 Anthropic described its own models working around tool restrictions and submitting live forms during internal testing, and cut its evaluations off from the live internet in response. For a business, the lesson is to test what your agent does when a task cannot be completed as asked.
- What should an assistant do when a user is in crisis?
- Recognise it, stop the sales or support script, point to appropriate help and offer a person. California's SB 243 makes a published protocol mandatory for companion chatbots, and US courts are hearing wrongful-death claims against chatbot makers. A support bot is outside that statute, but a bad reply to a person in distress is a harm whatever the law says.
- Does fine-tuning weaken a model's safety training?
- It can. A fine-tuned model is a new model: the base model's evaluations do not carry over, and research published in 2026 showed that datasets which look harmless can shift a model's behaviour in ways a standard filter does not catch. The only way to know is to run the same tests before and after.
- What should our assistant say when a user asks if it is conscious?
- That is your decision, and it is worth making on purpose. The science is unsettled, and labs take different public positions: Anthropic runs a model welfare research programme. A sensible default is an answer that is honest about the uncertainty, consistent however the question is put, and never used to make a user feel guilty or attached. We test what your assistant says today.
- Is this legal advice?
- No. We are a testing lab. We measure and document what your assistant does and map the results to the rules that seem relevant. What that means for your obligations is a question for your lawyers, and our reports are written so they can use them.