hematteo's picture
Serve the atlas from matteohe.com/learning-to-read-out
b58ffa7 verified
|
Raw History Blame Contribute Delete
1.09 kB
metadata
title: Readout Feature Atlas
emoji: 📖
colorFrom: blue
colorTo: indigo
sdk: static
pinned: true
license: mit
short_description: Browse readout features across Pythia pretraining
tags:
  - interpretability
  - mechanistic-interpretability
  - crosscoder
  - sparse-autoencoder
  - pythia
  - training-dynamics

Readout Feature Atlas

Interactive atlas of the parameter-trajectory crosscoders from Learning to Read Out: Unembedding Dynamics in Language Model Pretraining (NeurIPS 2026, arXiv:2610.21025).

Each feature of the Pythia-160M, 1B and 6.9B readout dictionaries has a page with its decoder-norm trajectory over 32 training checkpoints, its top tokens at checkpoints across training, its lifecycle profile, and its closest matches in the other model sizes.

This Space shows the atlas hosted at https://matteohe.com/learning-to-read-out/atlas/.