Article
Seeing ViT Registers: Model Size, LayerNorm, and Where the High-Norm Tokens Go
turhancan97
• • 1
🤝 Open to Collab
Turhan Can Kargın
turhancan97
AI & ML interests
Computer Vision, Deep Learning, Machine Learning, Robotics, Reinforcement Learning, Control Theory
Recent Activity
liked a model 2 days ago
mistralai/Mistral-Large-4-1T-A52B posted an update 5 days ago
New blog post: Seeing ViT Registers 🔍
I reproduced the main observation of "Vision Transformers Need Registers" (Darcet et al.) on six DINOv2 checkpoints from the Hub: Base, Large and Giant, each with and without registers.
Three findings:
1️⃣ Model size matters. In DINOv2 Giant, ~1.6% of patch tokens form a separate high-norm group. In Large the effect is weak, and in Base I found none.
2️⃣ Where you measure matters. The artifact tokens have very high norms before the final LayerNorm (hidden_states[-1]), but the lowest norms after it (last_hidden_state).
3️⃣ Registers take over. With registers, patch maps are smooth, and one register token carries a norm of ~1,346 instead.
Everything is inference-only and reproducible with one command. The post also includes practical tips for people who use DINOv2 features for dense tasks or probing.
📝 Blog: https://huggingface.co/blog/turhancan97/seeing-vit-registers
💻 Code: https://github.com/turhancan97/seeing-vit-registers upvoted an article 5 days ago
Seeing ViT Registers: Model Size, LayerNorm, and Where the High-Norm Tokens GoOrganizations
published an article 5 days ago
published an article 4 months ago
Article
StripeCycle: A CycleGAN You Can Actually Read — From-Scratch PyTorch, Start to Finish
turhancan97
• • 1 published an article about 3 years ago
Article
Building Your First Kubeflow Pipeline: A Comprehensive Guide
turhancan97
• • 4