On Agnostic PAC Learning in the Small Error Regime

TL;DR

Binary classification in the classic PAC model exhibits a curious phenomenon: Empirical Risk Minimization (ERM) learners are suboptimal in the realizable case yet optimal in the agnostic case…

Opening excerpt from the authors’ abstract. source