What if your AI model’s experts weren’t just the smartest, but the most diverse? Researchers from Renmin University, Huawei, and Tianjin University introduce MP-MoE: a routing method that selects diverse experts using co-occurrence patterns. Result: 1-3% boost in LLM
MP-MoE: Diverse Expert Routing Boosts Large Language Models
By
–
