When AI Rewrites the Internet, What Do We Lose?

A phenomenon called "model collapse" can occur when AI models are trained on data produced by earlier versions of themselves, leading to information loss, especially in data distribution tails. Two types are identified: "early model collapse" where data tails are lost, and "late model collapse" where models converge on a narrow, inaccurate distribution. Research shows even small amounts of synthetic data can distort models, reducing output diversity. This poses risks as AI-generated content becomes more prevalent in training datasets.