Generalized Gradient Norm Clipping & Non-Euclidean $(L_0,L_1)$-Smoothness

TL;DR

This work introduces a hybrid non-Euclidean optimization method which generalizes gradient norm clipping by combining steepest descent and conditional gradient approaches. The method achieves the best of both worlds by establishing a descent property under a generalized notion of ($L_0$,$L_1$)-smoothness.

Opening excerpt from the authors’ abstract. source