Revela: Dense Retriever Learning via Language Modeling

TL;DR

Dense retrievers play a vital role in accessing external and specialized knowledge to augment language models (LMs). Training dense retrievers typically requires annotated query-document pairs, which are costly to create and scarce in specialized domains (e.g., code) or in complex settings (e.g., requiring reasoning).

Opening excerpt from the authors’ abstract. source