Do you think the synthetic data is relevant here? What I mean is that some effective compute has been embedded into the data to generate these high quality tokens, so you don't need as much compute to get value out of them
Synthetic Data Quality Reduces Compute Requirements
By
–