We are honored to have met KAUST's President, Dr. Tony F Chan, and to work on revolutionary research together. At #SC23, King Abdullah University of Science and Technology (KAUST) and Cerebras Systems were finalists for the 2023 Gordon Bell Prize, the most prestigious award for
@cerebras
-
Modular Library Advances Sparse Machine Learning Research
By
–
(9/n) Our library is modular and extensible, enabling users to define algorithms without complex rewrites. It is already pivotal in supporting in-house research on sparsity, demonstrated through our works: *Sparse-IFT: https://
openreview.net/pdf?id=iP4WcJ4
EX0
…, and https://
proceedings.mlr.press/v216/thangaras
a23a.html
… * -
Cerebras Sparsity Optimization for Foundation Model Training
By
–
(10/n) Contact us to learn more about Cerebras and how sparsity can make training your next foundation model orders of magnitude more efficient. Shoutout to the amazing software, machine learning, and performance team members who’ve played an instrumental role in developing
-
RigL Dynamic Weight Sparsity Algorithm Implementation PyTorch
By
–
(8/n) These allow us to implement popular dynamic weight sparsity algorithms such as Rigging the Lottery (RigL) from @GoogleDeepMind
. Enabling these algorithms for any existing PyTorch model in our ModelZoo is as simple as making a few changes in our configuration file. -
Flexible Sparsity Distributions for Advanced Model Optimization
By
–
(7/n) We provide a lot of flexibility, including custom sparsity distributions, schedules for sparsity, and other richer features such as grouped tensor views and tie-breaking mechanisms.
-
PyTorch Sparse Optimizers for Efficient Model Training
By
–
(6/n) We rely on PyTorch’s optimizer interface to design all our sparse algorithms as special optimizers, which operate on a sparse view of the model parameters and gradients during training. We use a wrapper over existing PyTorch optimizers, which enables efficient
-
Library Enables Dense and Sparse AI Training Workloads
By
–
(4/n) Our library has been co-designed from the ground up to run both dense and #sparse training workloads. This enables practitioners to focus on what they do best: push the boundaries of AI research.
-
Cerebras CS-2 Accelerates Foundation Model Training Through Sparsity
By
–
(5/n) Our library is hardware agnostic, but combined with the Cerebras CS-2's unique ability to accelerate unstructured #sparsity, it unleashes unparalleled efficiency in training foundation models.
-
Sparsity Unlocks New ML Training Efficiency Dimension
By
–
(3/n) Sparsity helps unlock a new dimension of efficiency beyond model architectures for training, enabling control of #computing performance in the ML practitioner's hand. Sparse models also achieve better scaling but are difficult to accelerate. Today's #deeplearning libraries
-
Cerebras Releases PyTorch Sparsity Library for ML Researchers
By
–
(1/n) Excited to announce the "Cerebras PyTorch Sparsity Library," which democratizes access to #sparse training for all #ML researchers and developers. Read more here: https://
cerebras.net/blog/sparsity-
made-easy-introducing-the-cerebras-pytorch-sparsity-library
…