Rethinking Multimodal Learning from the Perspective of Mitigating Classification Ability Disproportion

TL;DR

Multimodal learning (MML) is significantly constrained by modality imbalance, leading to suboptimal performance in practice. While existing approaches primarily focus on balancing the learning of different modalities to address this issue, they fundamentally overlook the inherent disproportion in model classification ability, which serves as the pr…

Opening excerpt from the authors’ abstract. source