MoE very efficient and compress context KV store. Both noted in today’s post

→ View original post on X — @benthompson