TransMLA: Migrating GQA Models to MLA with Full DeepSeek Compatibility and Speedup

NeurIPSSpotlight2025

Authors: Fanxu Meng, Pingzhi Tang, Zengwei Yao, Xing Sun, Muhan Zhang
Venue: NeurIPS 2025
Track: Spotlight

TL;DR

Modern large-language models often face communication bottlenecks on current hardware rather than computational limitations…

Opening excerpt from the authors’ abstract. source

Read the paper

OpenReview

Topics

language model

← All NeurIPS 2025 Spotlight papers · Browse the whole archive