--- title: Readout Feature Atlas emoji: 📖 colorFrom: blue colorTo: indigo sdk: static pinned: true license: mit short_description: Browse readout features across Pythia pretraining tags: - interpretability - mechanistic-interpretability - crosscoder - sparse-autoencoder - pythia - training-dynamics --- # Readout Feature Atlas Interactive atlas of the parameter-trajectory crosscoders from **Learning to Read Out: Unembedding Dynamics in Language Model Pretraining** (NeurIPS 2026, [arXiv:2610.21025](https://arxiv.org/abs/2610.21025)). Each feature of the Pythia-160M, 1B and 6.9B readout dictionaries has a page with its decoder-norm trajectory over 32 training checkpoints, its top tokens at checkpoints across training, its lifecycle profile, and its closest matches in the other model sizes. This Space shows the atlas hosted at https://matteohe.com/learning-to-read-out/atlas/. - Project page: https://matteohe.com/learning-to-read-out/ - Code: https://github.com/hematteo/learning-to-read-out - Dictionaries: https://huggingface.co/hematteo/parameter-trajectory-crosscoders