There is now a smarter way to pick data for training LLMs! Enter OPUS! This is an ICML Oral paper from SJTU, Alibaba, UW–Madison, UIUC, and Mila – Quebec AI Institute. The proposed method dynamically and intelligently selects the most impactful data for LLM pre-training in
OPUS: Smarter data selection for LLM pre-training
By
–
