Synthetic data is a broad term. I think the paper looks at a very extreme case. One can blindly generate synthetic data by sampling from an upstream model. But yeah, I can see how being more strategic than that can actually be beneficial.

→ View original post on X — @rasbt,