We broke all records when we launched Cerebras Inference in August. Today we are tripling our performance from 650 t/s to 2100 t/s.
Cerebras Inference speed is in a league of its own – 16x faster than the fastest GPU solution, 68x faster than hyperscale clouds, and 4-8x faster
COMPUTING
-
Cerebras Triples Inference Speed to 2100 Tokens per Second
By
–
-
Cerebras Inference 3x Faster: Llama 70B Reaches 2,100 Tokens/Second
By
–
🚨 Cerebras Inference is now 3x faster:
— Cerebras (@cerebras) 24 octobre 2024
Llama3.1-70B just broke 2,100 tokens/s
– 16x faster than the fastest GPU solution
– 8x faster than GPUs running Llama *3B*
– It's like the perf of a new hardware generation in a single software release
Available now at… pic.twitter.com/9VgGWGO6qYCerebras Inference is now 3x faster: Llama3.1-70B just broke 2,100 tokens/s
– 16x faster than the fastest GPU solution
– 8x faster than GPUs running Llama *3B*
– It's like the perf of a new hardware generation in a single software release
Available now at -
Outlines Algorithms Ported to Rust for Faster Structured Generation
By
–
I've said it before: Structured generation is the future The @dottxtai team has ported its core Outlines algos to rust, which means from now on, structured LLM generation is: – faster
– more robust
– embeddable into any ML software (no need for Python anymore) ^#3 is my -
AI-Powered Cyber Resilience Transforms Cloud Security Strategy
By
–
Are you prepared for shifting from Continuous Threats to Continuous Business? I recently hosted Michael Fasulo at @Commvault on #CXOSpice. We explored "Cyber Resilience in the Cloud Era". Key insights include: Leveraging AI for faster threat detection and response
-
DuoAttention: Efficient Long-Context LLM Inference with Retrieval
By
–
DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads https://
arxiv.org/abs/2410.10819 https://
github.com/mit-han-lab/du
o-attention
… #MIT @songhan_mit -
MIT KV Cache Framework Reduces LLM Decoding Latency
By
–
MIT HAN Lab introduce a framework that only applies a full KV cache to retrieval heads while using a light-weight, constant-length KV cache for streaming heads, which reduces both LLM's decoding and pre-filling memory and latency without compromising its long-context abilities.
-
Neural and Non-Neural AI: Reasoning, Transformers, and LSTMs
By
–
Neural and Non-Neural AI, Reasoning, Transformers, and LSTMs https://
bit.ly/3XPO9aB
#AI #MachineLearning #DeepLearning #LLMs #DataScience -
Cerebras CrewAI NYC Hack Night Fast Inference Agents
By
–
Come hack with us! Join Cerebras and @crewAIInc for an NYC hack night next Wed, Oct 30. Explore what you can build with fast inference and AI agents RSVP here: https://
lu.ma/92uz16l4 -
VRAGE3 Game Simulation Scales Better CPU Core Saturation
By
–
In VRAGE3 the game simulation should scale better and saturate the CPU cores better.
-
SambaNova Cloud Achieves Fast Llama 3.2 Inference Performance
By
–
Hit the ground running with fast #AI inference on @AIatMeta
's Llama 3.2! SambaNova Cloud allows #devs to achieve 2470 tokens per sec on 1B and 1566 tokens per sec on 3B Start developing now