A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective

TL;DR

The widespread use of diffusion models has led to an abundance of AI-generated data, raising concerns about model collapse---a phenomenon in which recursive iterations of training on synthetic data le…

Opening excerpt from the authors’ abstract. source