Microsoft’s new 13B Orca 2 outperforms 70B models on many evals. Interesting synthetic instruct data:
1) try many detailed system prompts
2) generate answers using best prompt for each task
3) replace with generic system prompt in tuning demonstrations
Microsoft’s Orca 2 13B beats 70B models with novel synthetic instruct data approach
By
–
