wow, mixtral 8×7 can hit ~488 tok/s using groqchat's custom chips. realtime analysis or nested llm calls become way more feasible at these speeds

whole responses in about a second: pic.twitter.com/wv8jX4Uj93

— Aaron Ng (@localghost) 1 février 2024

wow, mixtral 8×7 can hit ~488 tok/s using groqchat's custom chips. realtime analysis or nested llm calls become way more feasible at these speeds whole responses in about a second:

→ View original post on X — @localghost,