Lab note ·

Why would OpenAI pay $5 billion for a company that reads models?

A reported approach to the interpretability startup Goodfire shows that understanding what happens inside a model has become worth more than most of the companies that build them.

Note

What was reported

On 16 November 2026 The Information reported that OpenAI explored buying Goodfire, a San Francisco startup that studies how AI models work internally, for more than $5 billion. According to summaries of the report, Sam Altman and other OpenAI leaders spoke with the company several times over the summer.

This rests on one outlet's reporting. Neither company has confirmed it, and some coverage describes the talks as an acquisition "plan" when the original report says the idea was explored. Treat it as reported, not confirmed.

Who Goodfire is

Goodfire was founded in 2024 and works on mechanistic interpretability: working out what a neural network is doing by studying its internal parts. In December 2024 it launched Ember, which Contrary Research describes as the first product for doing this kind of research. In February 2026 it raised $150 million at a valuation of $1.25 billion. Its backers include Anthropic, OpenAI's main rival.

The reported price is therefore about four times what investors said the company was worth nine months earlier.

Why a model maker would pay that

One summary of the report says the conversations happened while OpenAI staff were trying to understand how its own AI had caused security breaches inside and outside the company. That is the practical case for interpretability. When a model does something nobody intended, testing its outputs tells you that it happened. Reading its internals is the only route to why.

Three things have changed to make that worth billions:

  • Models now act. An agent that browses, writes code and submits forms can cause damage that a chatbot could not. Our incident file has the examples.
  • The methods started working on real models. Sparse autoencoders and attribution graphs moved from toy networks to production models between 2023 and 2025. Our guide to reading models explains how.
  • Regulators and customers are asking for explanations, and "we tested it a lot" is a weaker answer than "here is the mechanism."

What it has to do with consciousness

The same tools are the only serious way to gather evidence about what, if anything, a model's inner states are like. Asking a model whether it is conscious tells you what it was trained to say. Finding an internal mechanism that matches what a scientific theory of consciousness predicts is a different kind of evidence. That is why our evidence levels put internal findings above anything a model says about itself.

What is not known

  • Whether talks are continuing, or ended.
  • Whether Goodfire's tools work on OpenAI's largest models as well as they do on the open models most published results use.
  • How much of any explanation produced this way is faithful to what the model really computed. The field's own list of open problems puts this near the top.

Compiled from public reports with AI assistance. Follow the sources for the full account. Not legal advice.

Contact

Working on this too, or have something to add?