In the latest round of #MLPerf industry benchmarks, the first submission of NVIDIA Blackwell delivered up to 4X more performance on #Llama 2 70B, while the NVIDIA H200 Tensor Core GPU continued to deliver outstanding results across every data center test.
COMPUTING
-
UAI Breaks MLPerf Inference Records with speedAI240 Cards
By
–
UAI sets new records in @MLCommons #MLPerf® Inference v4.1 benchmarks with our speedAI®240 cards: 6x energy efficiency vs comp. Highest single-card throughput 3x lower latency vs comp. Fastest ever recorded latency for ResNet-50 Learn more: https://
untether.ai/mlperf-results/ -
Improving Landing System Reliability Through Iterative Failure Analysis
By
–
Now we figure out what went wrong to drive the landing failure rate far above 1 in a thousand, then 1 in 10 thousand … 1 in a milion, etc
-
India’s AI Compute Capacity Initiative with 10,000 GPUs
By
–
The IndiaAI Compute Capacity pillar of the INDIAai mission focuses on creating a scalable AI computing ecosystem with over 10,000 GPUs through public-private partnerships to meet the demands of India's growing AI startups and research community.
— IndiaAI (@OfficialINDIAai) 28 août 2024
The marketplace will offer AI as a… pic.twitter.com/xAR9zNb8vKThe IndiaAI Compute Capacity pillar of the INDIAai mission focuses on creating a scalable AI computing ecosystem with over 10,000 GPUs through public-private partnerships to meet the demands of India's growing AI startups and research community.
The marketplace will offer AI as a -
Cerebras Debuts World’s Fastest Inference Technology
By
–
It was an interesting couple of days hearing from Cerebras about their move into inference. I think the company made a compelling pitch for different technology needed to enable different kinds of Gen AI queries. AI startup Cerebras debuts ‘world’s fastest inference,’ with a
-
Deploy Optimized LLMs in Your Private Cloud Securely
By
–
What if you could have your own highly-optimized #LLMs running in your #private cloud without any hassle? Well now you can. No more choosing between #performance and #security — have your LLM cake and eat it too! Want to learn how? Save a spot for our webinar
-
SambaNova Achieves 114 Tokens Per Second Record with Llama
By
–
I've been playing with @SambaNovaAI
's API serving fast Llama 3.1 405B tokens. Really cool to see leading model running at speed. Congrats to Samba Nova for hitting a 114 tokens/sec speed record (and also thanks @KunleOlukotun for getting me an API key!) -
Cerebras Inference Achieves Record Throughput for Llama 3.1 Models
By
–
Verified by @ArtificialAnlys, @CerebrasSystems Inference is capable of serving Llama 3.1 70B at 450 tokens/sec and Llama 3.1 8B at 1,850 tokens/sec! https://t.co/hCb9MmSvOo
— AI at Meta (@AIatMeta) 27 août 2024Verified by @ArtificialAnlys
, @cerebras Inference is capable of serving Llama 3.1 70B at 450 tokens/sec and Llama 3.1 8B at 1,850 tokens/sec! -
API Cache Storage Costs and TTL Configuration Explained
By
–
Thanks for sharing! My first question was how much does it cost for cache storage and how does that work. I found the answer at the bottom of the api docs so hopefully this helps others with the same question. No extra storage costs and cache has a ~5 min TTL.