Reading models · Inside view

Watching a model work, one word at a time.

Our consciousness tests ask models questions and score what they say. Circuit tracing does something different: it follows what happens inside the model on the way to an answer. It is the nearest thing there is to checking a model's account of itself against what it actually did.

What it is

A language model produces one word at a time. Between reading your text and choosing that word, millions of internal units switch on and off. Circuit tracing picks out the units that mattered for one particular word and draws how they fed into each other. The drawing is called an attribution graph.

Anthropic published the method in March 2025 and used it on its own Claude models. Three findings got the attention. When writing rhyming verse, the model chose the rhyme word in advance and then wrote the line to reach it. When answering a question that needs two facts, it could be seen combining the first fact with the second rather than recalling a memorized answer. And on a hard sum where it was given a hint, the steps it wrote out did not match what it had done inside: it worked backward from the hinted answer and presented that as a calculation.

That last one matters most for anyone asking whether a model's statements about itself can be trusted. A model's explanation of its own reasoning is another piece of generated text. Sometimes it describes the real process and sometimes it does not, and from the outside the two look the same.

What anyone can do

In May 2025 the method was released as open-source software, written by Anthropic Fellows with Decode Research. It runs on open-weight models, the ones whose internals are published, and a free website, Neuronpedia, hosts it so no hardware is needed.

You want toWhat it takesWorks on
Trace a sentence in your browserNothing. Type a prompt on Neuronpedia's tracer ↗ and wait a few seconds.Small open models. The first release covered Gemma-2-2b and Llama-3.2-1b; more have been added since.
Run your own experimentsThe circuit-tracer library ↗ on a rented graphics card, and the patience to read graphs. It also lets you turn a feature up or down and watch the answer change.Open-weight models that have the needed add-on (a “transcoder”) trained for them.
Trace GPT, Gemini or ClaudeNot possible from outside. The internals of these models are not published; only their makers can look.Nobody but the lab that built the model.

So this is not hidden behind billions, but it is limited to small open models. Findings on a two-billion-parameter model do not automatically hold for the largest ones.

How this fits our tests

The models in our results are closed, so we cannot trace them. What tracing gives us is a check on the method itself: on models small enough to open, we can see what stands behind a “yes” or a “no” to the same kinds of question we put to the large ones, and compare what a model says with what drove it to say it. Where the two disagree on small models, a claim made by a large one deserves the same doubt.