Ah yeah, and the notebook if you prefer a from-scratch approach: https://
github.com/rasbt/LLMs-fro
m-scratch/blob/main/ch07/04_preference-tuning-with-dpo/dpo-from-scratch.ipynb
…

→ View original post on X — @rasbt