(9/n) Our library is modular and extensible, enabling users to define algorithms without complex rewrites. It is already pivotal in supporting in-house research on sparsity, demonstrated through our works: *Sparse-IFT: https://
openreview.net/pdf?id=iP4WcJ4
EX0
…, and https://
proceedings.mlr.press/v216/thangaras
a23a.html
… *
CODE
-
Modular Library Advances Sparse Machine Learning Research
By
–
-
Flexible Sparsity Distributions for Advanced Model Optimization
By
–
(7/n) We provide a lot of flexibility, including custom sparsity distributions, schedules for sparsity, and other richer features such as grouped tensor views and tie-breaking mechanisms.
-
RigL Dynamic Weight Sparsity Algorithm Implementation PyTorch
By
–
(8/n) These allow us to implement popular dynamic weight sparsity algorithms such as Rigging the Lottery (RigL) from @GoogleDeepMind
. Enabling these algorithms for any existing PyTorch model in our ModelZoo is as simple as making a few changes in our configuration file. -
PyTorch Sparse Optimizers for Efficient Model Training
By
–
(6/n) We rely on PyTorch’s optimizer interface to design all our sparse algorithms as special optimizers, which operate on a sparse view of the model parameters and gradients during training. We use a wrapper over existing PyTorch optimizers, which enables efficient
-
Library Enables Dense and Sparse AI Training Workloads
By
–
(4/n) Our library has been co-designed from the ground up to run both dense and #sparse training workloads. This enables practitioners to focus on what they do best: push the boundaries of AI research.
-
Cerebras Releases PyTorch Sparsity Library for ML Researchers
By
–
(1/n) Excited to announce the "Cerebras PyTorch Sparsity Library," which democratizes access to #sparse training for all #ML researchers and developers. Read more here: https://
cerebras.net/blog/sparsity-
made-easy-introducing-the-cerebras-pytorch-sparsity-library
… -
Hugging Face Competitions: New Features and Enhanced Customization Options
By
–
New Features in Hugging Face Competitions – support for multiple metrics
– custom metrics
– fully hidden test set
– custom hardware support: CPU, A10g, A100
– custom requirements support
– custom time-limit for submission runs
– support for not just submission.csv but any -
Corrective RAG: Self-Reflection Improves Retrieval Quality
By
–
Corrective RAG Corrective RAG (CRAG) is a recent paper that uses self-reflection to identify and correct problems in retrieval. It first uses a retrieval evaluator to assess the quality of retrieved documents relative to the query. It filters out irrelevant documents and
-
LlamaIndex Roadmap: New LLM Models, Vector Stores, Ingestion
By
–
Brief roadmap for more LlamaIndex integrations: More LLM/Embedding models More vector stores Ingestion Pipeline Transformation We'll be doing a live webinar on 16th with @llama_index for more in-depth walkthrough. Have more ideas? Let us know below