My intuition is two things:
1. Carefully curated pre-training data & finding the right data mix 2. Domain/ task specific evals for downstream use-cases In the end training data is still king! – finding which combination to go for is where the real moneys at. In addition- Post

→ View original post on X — @reach_vb,