It's terrible for most regular people. We are memory poor — MoE models make this even worse. It may be cute, but it's totally impractical. Dense models are far more usable in practice.

→ View original post on X — @jeremyphoward,