The whole Reflection-70B debacle points the the desperate need for a better AI evaluation ecosystem. It needs to be extremely easy to adjudicate:
(1) is the model overfit to benchmarks
(2) is the model truly unique (i.e. not a wrapper or thin fine-tune) https://
x.com/shinboson/stat
/shinboson/status/1832933747529834747
…
AI Evaluation Ecosystem Needs Better Benchmark Overfitting Detection
By
–