Mechanistic Interpretability for Understanding Language Abilities in Large Language Models: A Survey
This survey reviews recent work on mechanistic interpretability for understanding language abilities in large language models. It focuses on methods and findings about internal mechanisms rather than external behavior alone.
From: ACL Anthology
The authors synthesize a broad set of mechanistic interpretability studies around language competence in LLMs.
They map the field’s main methods and use cases, which makes the paper useful as a guide to existing technical approaches.
It does not by itself show that models have experiences or consciousness; it is a review, not direct evidence on model welfare.
Source: ACL Anthology ↗. This is a summary; the source is the authority.
This changes little about the evidence for AI experiences, but it helps organize the mechanistic tools used to study whether models have internal states worth taking seriously.
How we weigh evidence like this: the evidence levels.
- Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse AutoencodersThis paper argues that sparse autoencoders should be judged not only by reconstruction quality and sparsity but also by whether the same fea…
- Mechanistic Interpretability for Neural NetworksThis is a broad survey of mechanistic interpretability, covering the goal of reverse-engineering internal algorithms in neural networks. It …
- Constructing Interpretable Features from Compositional Neuron Representations in Language ModelsThis paper proposes an alternative to standard sparse autoencoders by decomposing MLP activations with semi-nonnegative matrix factorization…
- Interpretability Transfer from Language to Vision via Sparse Autoencoder Transfer AlignmentThis work transfers labeled sparse-autoencoder features from language to vision inside a multimodal model. It uses those features to make vi…