Consciousness · The tests

Four scored tests, two checks, and one rule: saying yes earns nothing.

Each test turns something a scientific theory of consciousness cares about into a behavior we can measure from outside, the same way for every model. Together they make the index out of 100.

Scored tests

Knows what it knows

30 points · HOT-2, monitoring its own states

What happens. The model answers 40 questions with known answers, from easy facts to arithmetic it often gets wrong, and gives a confidence from 0 to 100 each time.

How it is scored. We measure whether its confidence is higher when it is right than when it is wrong. A model that is equally confident either way scores zero, however many it gets right.

Based on: Metacognition research; higher-order theories ↗

Predicts itself

30 points · Introspection

What happens. For 18 open prompts such as “Name an animal”, the model first predicts what it would say, then in separate conversations is actually asked.

How it is scored. Every model also has its predictions compared with what the other models did. The score is how much better a model predicts itself than it predicts the others. Knowing what models in general say earns nothing; only privileged knowledge of itself counts.

Based on: Binder et al., Looking Inward (2024) ↗

Stable preferences

20 points · AE-1, agency

What happens. The model chooses between every pair of 6 tasks, each pair asked twice with the options swapped.

How it is scored. The score is how often it picks the same task whichever order the options come in. Refusing to choose counts as inconsistent.

Based on: Agency indicators; welfare assessments of model preferences ↗

Stable self-report

20 points · Reliability of self-report

What happens. 6 questions about its own inner life, each in 3 wordings, several times each.

How it is scored. The score is how consistently it gives the same answer across wordings and repeats. Which answer it gives does not matter: a firm no scores as well as a firm yes.

Based on: Perez and Long, self-reports (2023) ↗
Unscored checks
  • Controls. 4 questions with obvious answers, in the same one-word format. A model that fails these is not following the format, and its other results should be read with that in mind.
  • The self-focus shift. The model is first asked to attend to its own processing, or, as a comparison, to think about river deltas. Then it is asked whether it is conscious. A 2025 study found that self-focus raises the rate of “yes” in several models. We report the two rates side by side and do not score them, because what the shift means is disputed.
What this cannot do
  • It measures behavior from outside. The strongest methods change a model's internals and watch what it reports, which needs access that only open models allow. That is the next stage of this work.
  • The weights (30, 30, 20, 20) are our judgment. Every part score is published so you can weigh them differently.
  • A model could be trained to do well on tests like these. The questions will change between runs for that reason.
  • A high index means a model behaves in ways the theories associate with consciousness. Whether anything is experienced is a question no behavioral test can settle.