MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction

TL;DR

Universal multimodal embedding models have achieved great success in capturing semantic relevance between queries and candidates. However, current methods either condense queries and candidates into a single vector, potentially limiting the expressiveness for fine-grained information, or produce too many vectors that are prohibitively expensive...

Opening excerpt from the authors’ abstract. source