Semi-Supervised Preference Optimization with Limited Feedback

TL;DR

The field of preference optimization has made outstanding contributions to the alignment of language models with human preferences. Despite these advancements, recent methods still rely heavily on substantial paired (labeled) feedback data, leading to substantial resource expenditures.

Opening excerpt from the authors’ abstract. source