Good catch! This likely would be fixed with more robust training (more steps + rollouts, and ideally a larger than 1.5B model) 🙂 This was mostly just to show how it works. Feel free to try the same task with a larger model, more steps etc.

→ View original post on X — @mattshumer_