AutoTrain just added LLM ORPO as a task! ORPO provides the same performance as DPO with 50% reduction in memory since no reference model is needed Now, train your own zephyr-orpo like models without writing a single line of code [powered by trl]

→ View original post on X — @abhi1thakur,