You can also find a HellaSwag evaluation script that works with one large batch item, EOT separators, and document_mask here: https://
github.com/alexjc/nanogpt
-speedrun/blob/master/eval_gpt2.py
… It's setup to use TokenMonster too, so you can check the scores on downstream benchmarks easily hopefully.

→ View original post on X — @alexjc