You don't need RLHF for stop tokens; GPT‑3 had that long before RLHF shipped, SFT works fine, and here base Llama 3.1 gets it (presumably) just from Llama 2 prompts floating around online. RLHF does a lot but it's not that deep; I doubt it's adding new concepts.

→ View original post on X — @goodside