When you’re pre-training, you basically learn from everything in the dataset. When fine-tuning, you often only want the model to learn from the text in the output, not the prompt itself.
Pre-training vs Fine-tuning: Learning Strategies in AI Models
By
–