I’ve never tried it or had a need to. If you write good prompts, you don’t need to worry about the model going off the rails.
@mattshumer_
-
Axolotl and Mistral for Synthetic Data Generation
By
–
Axolotl. Fucking awesome. Shoutout to @winglian
. For synthetic data, Mistral models are great. Pair that with some real-world data for grounding/diversity/inspiration for good results. @csahil28 is building an even better solution here. Costs depend on the task at hand. Can be -
Fine-tuning Open-ended LLM Agents: Challenges and Personality Experiments
By
–
The most fun, but frustrating to do (due to data challenges) are open-ended agent fine-tunes. Easier ones are more chat/tool-use focused. A really fun one was experimenting with changing a LLM’s personality significantly. Made it “sassy” lol
-
Pre-training vs Fine-tuning: Learning Strategies in AI Models
By
–
When you’re pre-training, you basically learn from everything in the dataset. When fine-tuning, you often only want the model to learn from the text in the output, not the prompt itself.
-
LLM Prompting and Fine-tuning Questions and Answers
By
–
Ask me your LLM prompting/fine-tuning questions. Have an hour, will answer as many as I can.
-
Upcoming open-source project launch announcement
By
–
May do this next week! Will open-source of course
-
Mistral 7B Improvements Enable Better Fine-Tuning Capabilities
By
–
This is huge news. Mistral 7B was already the best model in its size class, and these improvements are a huge step up. I’ll be re-training many of my current fine-tunes over this model ASAP.
-
Better LLM Reward Model Performance Without Numerical Scales
By
–
If you’re using out of the box LLMs as a reward model/sample rater, here’s a trick to get way better performance: Don’t ask the LLM to rate examples on a numerical scale (i.e. 1-5). The model will almost always choose 1 or 5. Instead, use words as rating options (“very bad”,
-
Axolotl and Together Compute for Model Fine-tuning
By
–
I use Axolotl, but services like @togethercompute might be easier. Last I checked, they don’t support only training on outputs, which is important. May have changed since!
-
Fine-tuning LLMs: Progressive Data Selection Strategy
By
–
High alpha technique for fine-tuning LLMs: First, fine-tune your model on all your data Then, use AI to pick a smaller set of really good examples, and tune on them Finally, hand-pick and tune on a few hundred incredible examples >> Huge performance gains <<