Multilingual Sparse Autoencoders and Principled Layer Selection for Cross-Lingual Steering in Large Language Models
This paper studies sparse autoencoders for feature-level interpretability and steering in multilingual language models. It also proposes a more principled way to choose which layers to target for control.
From: ACL Anthology
The authors test SAE-based language control in multilingual settings where English-only SAEs often transfer poorly.
They show that steering quality depends heavily on layer choice and multilingual training choices.
It does not demonstrate self-introspection, accurate report of internal states, or anything about consciousness.
Source: ACL Anthology ↗. This is a summary; the source is the authority.
This adds evidence that SAE-based control can work across languages, but it does not change the evidential status of model experiences.
How we weigh evidence like this: the evidence levels.
- Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse AutoencodersThis paper argues that sparse autoencoders should be judged not only by reconstruction quality and sparsity but also by whether the same fea…
- Mechanistic Interpretability for Neural NetworksThis is a broad survey of mechanistic interpretability, covering the goal of reverse-engineering internal algorithms in neural networks. It …
- Constructing Interpretable Features from Compositional Neuron Representations in Language ModelsThis paper proposes an alternative to standard sparse autoencoders by decomposing MLP activations with semi-nonnegative matrix factorization…
- Interpretability Transfer from Language to Vision via Sparse Autoencoder Transfer AlignmentThis work transfers labeled sparse-autoencoder features from language to vision inside a multimodal model. It uses those features to make vi…