15/ To make sure that the model does as few as possible of these sort of errors, we can use “few shots”: we provide the model with a few examples in the prompt with the expected answers. Since this improves performance, MMLU is evaluated in 5 shots in all our experiments

→ View original post on X — @thom_wolf,