Fast Escape, Slow Convergence: Learning Dynamics of Phase Retrieval under Power-Law Data

TL;DR

Scaling laws describe how learning performance improves with data, compute, or training time, and have become a central theme in modern deep learning. We study this phenomenon in a canonical nonlinear model: phase retrieval with anisotropic Gaussian inputs whose covariance spectrum follows a power law.

Opening excerpt from the authors’ abstract. source