What do you mean with set of prompts? A LLM judge with a rubric? (That’s a different approach; covered in the appendix).
The reason for a symbolic verifier is that it is symbolic. There is no nondeterminism, bias, etc. Doesn’t work for everything, but eg for math there is no

→ View original post on X — @rasbt,