I think synthetic data for fine-tuning is a seoarate issue from whether accidental AI-generated text in pre-training data has an impact on model quality

→ View original post on X — @simonw