ZeroS: Zero‑Sum Linear Attention for Efficient Transformers

NeurIPSSpotlight2025

Authors
Jiecheng Lu, Xu Han, Yan Sun, Viresh Pati, Yubin Kim, Siddhartha Somani, Shihao Yang
Venue
NeurIPS 2025
Track
Spotlight

TL;DR

Linear attention methods offer Transformers $O(N)$ complexity but typically underperform standard softmax attention…

Opening excerpt from the authors’ abstract. source

Read the paper

Topics

transformer attention efficient

Related NeurIPS papers

← All NeurIPS 2025 Spotlight papers · Browse the whole archive