Great LLM Inference benchmark — all the right variables are in there.
Now, the race to the bottom begins, because in LLM Inference there isn't a sustainable long-term technology or software moat. IMHO the margins are going to resemble grocery stores and laundromats…
Some of… https://t.co/pR5xVIpik3

— Soumith Chintala (@soumithchintala) 26 janvier 2024

Great LLM Inference benchmark — all the right variables are in there.
Now, the race to the bottom begins, because in LLM Inference there isn't a sustainable long-term technology or software moat. IMHO the margins are going to resemble grocery stores and laundromats…
Some of

→ View original post on X — @soumithchintala