Super funny to see an official Phi MoE! Phixtral is a little project I made 8 months ago by combining 2-4 finetunes with MergeKit. It worked better than expected. The Phi-3.5 MoE looks great on benchmark, curious to see how it performs in practice. Model:
Phi-3.5 MoE: Official Mixture of Experts Model Performance
By
–
