MoBA: Mixture of Block Attention for Long-Context LLMs

NeurIPSSpotlight2025

Authors: Enzhe Lu, Zhejun Jiang, Jingyuan Liu, Yulun Du, Tao Jiang, Chao Hong, Shaowei Liu, Weiran He, Enming Yuan, Yuzhi Wang, Zhiqi Huang, Huan Yuan, Suting Xu, Xinran Xu, Guokun Lai, Yanru Chen, Huabin Zheng, Junjie Yan, Jianlin Su, Yuxin Wu, Yutao Zhang, Zhilin Yang, Xinyu Zhou, Mingxing Zhang, Jiezhong Qiu
Venue: NeurIPS 2025
Track: Spotlight

TL;DR

Scaling the effective context length is essential for advancing large language models (LLMs) toward artificial general intelligence (AGI)

Opening excerpt from the authors’ abstract. source

Read the paper

OpenReview

Topics

large language model language model attention llm

← All NeurIPS 2025 Spotlight papers · Browse the whole archive