They really need KV caching though – otherwise the pricing is just not competitive. (OTOH if they *do* add caching at a good price point, they will be amazing!)

→ View original post on X — @jeremyphoward,