I've been playing with @SambaNovaAI
's API serving fast Llama 3.1 405B tokens. Really cool to see leading model running at speed. Congrats to Samba Nova for hitting a 114 tokens/sec speed record (and also thanks @KunleOlukotun for getting me an API key!)
SambaNova Achieves 114 Tokens Per Second Record with Llama
By
–