I fine-tuned Mistral7Bv0.2 locally with MLX using this DPO dataset by @argilla_io
, composed of 7k chat interactions distilled from Capybara. I haven't done a full evaluation yet, but for this size, it's really good, and it understood this famous question right away! Link
Fine-tuning Mistral 7B with DPO dataset for improved chat performance
By
–
