Create your Phi MoE In January, I released Phixtral, the first MoE based on Phi-2. I said I would release the code, but Microsoft decided to modify Phi-2, which broke my version. Thanks to Vishal Rajendra Kumar, we now have a way to reproduce Phixtral with his Phi2xtral

→ View original post on X — @maximelabonne,