Learn more here: https://
cerebras.ai/press-release/
cerebras-demonstrates-trillion-parameter-model-training-on-a-single-cs-3-system
… https://
cerebras.ai/blog/introduci
ng-gigagpt-gpt-3-sized-models-in-565-lines-of-code
…
HARDWARE
-
Cerebras Trains Trillion-Parameter Models on Single CS-3
By
–
-
Cerebras-GPT vs Megatron: Simplifying GPU Training Architecture
By
–
Nvidia is very proud of Megatron – it lets you train across thousands of GPUs! But it's 20K lines of code to manage the cluster. Cerebras-GPT is 500 lines of code. One block of memory. One logical accelerator. No distributed computing. Everyone who's used it calls it magic.
-
MemoryX Deploys 55TB DDR5 Memory in Commodity Server Format
By
–
MemoryX uses commodity DDR5 memory in a 1U server format, making it super easy to procure & configure. We employed one rack for CS-3 & networking and another rack for a 55 TB MemoryX partition.
-
Cerebras MemoryX: 55TB External Memory for Trillion Parameters
By
–
Cerebras Wafer Scale Cluster uses unique, terabyte-scale external memory device called MemoryX to store model weights. For Sandia’s run, Cerebras configured a 55 terabyte MemoryX device – enough to comfortably store 1T parameters and optimizer states.
-
Trillion Parameter Models Demand Massive GPU Infrastructure Investment
By
–
Trillion parameter models require terabytes of memory. Thousands of GPUs must be procured and connected just to store the model weights. It takes months to bring up a working cluster of this scale. Nvidia's own chart below:
-
Cerebras CS-3 trains 1 trillion parameter model efficiently
By
–
Cerebras Systems + Sandia National Labs have demonstrated training of a 1 trillion parameter model on a single CS-3 system (!) This is ~1% the footprint & power of an equivalent GPU cluster.
-
AmbiRobotics Achieves 500 Sorts Per Hour Speed Milestone
By
–
In 2024, @AmbiRobotics “increased avg speed to 500 sorts per hour and brought its sort accuracy to 99.6%. For context, in 2022 their machines were at 355 sorts per hour and in 2023 they were at 410.”
-
Elon Musk and Google Plan Giant Quantum Computer in Space
By
–
Pendant que les politiciens français se déchirent sur des affaires de corneculs debiles @elonmusk et @sundarpichai le patron de @google prévoient de construire un ordinateur quantique géant dans l’espace LA FRANCE VA COULER
-
Meta Quest 3, Quest 3S and Solos AirGo VR Hardware
By
–
Meta Quest 3, Quest 3S and Solos AirGo are pretty cool
-
GPU Availability Challenges for AI Projects
By
–
I have an open thread on slack – if the GPU gods allow, then we shall!