Fwiw, Big-bench actually proposed new novel tasks. It's not a repackage. Aggregate results can be broken in many ways as mentioned in the benchmark lottery paper.

→ View original post on X — @yitayml