On Linear Mode Connectivity of Mixture-of-Experts Architectures

NeurIPSOral2025

Authors
Viet-Hoang Tran, Van-Hoan Trinh, Khanh Vinh Bui, Tan Minh Nguyen
Affiliation
National University of Singapore
Venue
NeurIPS 2025
Track
Oral

TL;DR

We investigate Linear Mode Connectivity (LMC) in Mixture-of-Experts (MoE) architectures by analyzing their underlying permutation symmetries and proposing expert-matching algorithms that align independently trained MoEs to reveal LMC.

Opening excerpt from the authors’ abstract. source

Read the paper

Related NeurIPS papers

← All NeurIPS 2025 Oral papers · Browse the whole archive