In general, I'd recommend Unsloth for single-GPU settings and Axolotl for multi-GPU settings. For research, you might want to have more fine-grained control though. TRL works there, but reimplementing it in PyTorch is also a good option for full control.

→ View original post on X — @maximelabonne,