Research watch · Paper · Jul 2026

Constructing Interpretable Features from Compositional Neuron Representations in Language Models

This paper proposes an alternative to standard sparse autoencoders by decomposing MLP activations with semi-nonnegative matrix factorization. It argues that the resulting features are more directly tied to the model’s computations and can be easier to interpret causally.

Our reading

From: ACL Anthology

The authors compare their factorization approach with sparse autoencoders and a supervised baseline across several language models.

They report stronger causal steering performance and better alignment with human-interpretable concepts than the comparators.

It does not show that these features are introspective reports from the model itself, nor that causal interpretability implies subjective experience.

Source: ACL Anthology ↗. This is a summary; the source is the authority.

What it changes

This improves evidence that internal features can be extracted and manipulated, but it still falls short of evidence for experience or welfare.

How we weigh evidence like this: the evidence levels.