Congrats @andrewdfeldman and @CerebrasSystems for a huge leap forward and setting a new speed record for serving Llama 3.1-70B. 2100 tokens/sec is blazingly fast for a 70B model. This is great for agentic AI! https://t.co/LmIAXKQb3W
— Andrew Ng (@AndrewYNg) 26 octobre 2024
Congrats @andrewdfeldman and @cerebras for a huge leap forward and setting a new speed record for serving Llama 3.1-70B. 2100 tokens/sec is blazingly fast for a 70B model. This is great for agentic AI!