Depends on the LLM and its training data. Will have to think more about the second question.
LLMS
-
Fine-tune Mistral-7B with synthetic prompt-response datasets
By
–
Switch all prompts to mistral-large. Run the repo many times, collect the prompts / responses for each input / output. Format as JSONL, train a Mistral-7B or similar on all of them, and slot it in.
-
API Optimization and qLoRA Fine-tuning Techniques
By
–
On APIs, it’s hard but if you’re going after a specific use-case it just takes a bit of tinkering with the prompt. If not, just do a qLoRA with a couple hundred negative examples and you’re all set.
-
Prompt First: When to Fine-tune AI Models
By
–
Always, always try prompting first. The only time you should fine-tune is when you NEED to.
-
LLM API Strategy: Wait for PMF Before Fine-tuning
By
–
If the LLM API can do your task, if you can afford the costs, wait to fine-tune until you have PMF and need to optimize.
-
Chaining Smaller Prompts vs Mega Prompts Strategy
By
–
Usually stringing smaller prompt outputs together with code is better, but mega prompts are a close and easier second.
-
Fine-tuning AI models in 10 minutes with Runpod
By
–
10 minutes? Pretty doable actually, if the data is ready. Spin up a Runpod machine. While that’s starting, upload dataset to huggingface. Then write a yaml for the run config (Axolotl) and upload to Gist. By that time the machine is ready. Run this command: pip uninstall -y
-
Axolotl and Mistral for Synthetic Data Generation
By
–
Axolotl. Fucking awesome. Shoutout to @winglian
. For synthetic data, Mistral models are great. Pair that with some real-world data for grounding/diversity/inspiration for good results. @csahil28 is building an even better solution here. Costs depend on the task at hand. Can be -
Fine-tuning Open-ended LLM Agents: Challenges and Personality Experiments
By
–
The most fun, but frustrating to do (due to data challenges) are open-ended agent fine-tunes. Easier ones are more chat/tool-use focused. A really fun one was experimenting with changing a LLM’s personality significantly. Made it “sassy” lol
-
Pre-training vs Fine-tuning: Learning Strategies in AI Models
By
–
When you’re pre-training, you basically learn from everything in the dataset. When fine-tuning, you often only want the model to learn from the text in the output, not the prompt itself.