MonarchAttention: Zero-Shot Conversion to Fast, Hardware-Aware Structured Attention

TL;DR

Transformers have achieved state-of-the-art performance across various tasks, but suffer from a notable quadratic complexity in sequence length due to the attention mechanism…

Opening excerpt from the authors’ abstract. source