The benefits include: An essentially infinite number of problem instances of varying difficulty is readily available, strong baselines already exist, the evaluation metric (compression) cannot be gamed, and pretraining data contamination is highly unlikely.
Compression Benchmarks: Infinite Problem Instances for AI Evaluation
By
–
