Vision Transformers Don't Need Trained Registers

TL;DR

We investigate the mechanism underlying a previously identified phenomenon in Vision Transformers -- the emergence of high-norm tokens that lead to noisy attention maps (Darcet et al…

Opening excerpt from the authors’ abstract. source