Scaling Laws and Spectra of Shallow Neural Networks in the Feature Learning Regime
ICLROral2026
TL;DR
We derive a phase diagram of scaling laws for diagonal and quadratic neural networks via a bridge to LASSO and matrix compressed sensing, predicting both generalization and the emergence of power-law weight spectra.
Opening excerpt from the authors’ abstract. source
Read the paper
Topics
generalization scaling law