SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training

NeurIPSSpotlight2025

Authors
Jintao Zhang, Jia wei, Haoxu Wang, Pengle Zhang, Xiaoming Xu, Haofeng Huang, Kai Jiang, Jianfei Chen, Jun Zhu
Venue
NeurIPS 2025
Track
Spotlight

TL;DR

The efficiency of attention is important due to its quadratic time complexity…

Opening excerpt from the authors’ abstract. source

Read the paper

Topics

exploration attention

Related NeurIPS papers

← All NeurIPS 2025 Spotlight papers · Browse the whole archive