Vision Transformers with Self-Distilled Registers

TL;DR

Vision Transformers (ViTs) have emerged as the dominant architecture for visual processing tasks, demonstrating excellent scalability with increased training data and model size…

Opening excerpt from the authors’ abstract. source