We used two different techniques for quantizing these models.
Quantization-Aware Training with LoRA adaptorsprioritizing accuracy.
SpinQuant, a post-training quantization method which prioritizes portability.
Both versions are available for download as part of this release.
@aiatmeta
-
Two Quantization Techniques for Model Optimization Released
By
–
-
Llama 3.2 Quantized Versions Speed Up Inference 2-4x
By
–
We want to make it easier for more people to build with Llama — so today we’re releasing new quantized versions of Llama 3.2 1B & 3B that deliver up to 2-4x increases in inference speed and, on average, 56% reduction in model size, and 41% reduction in memory footprint.
Details -
Meta Movie Gen Research Paper and Media Foundation Models Breakdown
By
–
That’s all for now, but we’re excited to share more. You can see even more breakdowns and examples of what Meta Movie Gen can do in the full research paper https://
ai.meta.com/research/publi
cations/movie-gen-a-cast-of-media-foundation-models/?utm_source=twitter&utm_medium=organic_social&utm_content=thread&utm_campaign=moviegen
… -
Movie Gen Audio Output Demo Walk Through
By
–
Walk with us (and turn up your volume to hear the output from Movie Gen Audio in the last clip!) pic.twitter.com/w1rmLlDP6E
— AI at Meta (@AIatMeta) 23 octobre 2024Walk with us (and turn up your volume to hear the output from Movie Gen Audio in the last clip!)
-
Meta Movie Gen Models Deliver Impressive Results in Latest Examples
By
–
We’re continuing to see exciting results as we work with our Meta Movie Gen models, here are some more examples of what they can do 🧵 pic.twitter.com/Prswvvcc6v
— AI at Meta (@AIatMeta) 23 octobre 2024We’re continuing to see exciting results as we work with our Meta Movie Gen models, here are some more examples of what they can do
-
Meta Lingua: Lightweight Framework for Language Model Training
By
–
We recently released Meta Lingua, a lightweight and self-contained codebase designed to train language models at scale. Lingua is designed for research and uses easy-to-modify @PyTorch components in order to try new architectures, losses, data and more.
-
Knowledge Distillation with Llama 3.1 405B: Fine-tuning and Synthetic Data
By
–
New video – Knowledge Distillation with Llama 3.1 405B. In this extended discussion, @nazeri2010 & @subramen delve into the importance of fine-tuning — and how Llama 405B can be used to generate synthetic data. Full video on YouTube https://
youtu.be/kxx6TfpAkEU -
Research paper, model weights and code details released
By
–
More details, including links to the research paper, model weights and code
-
Meta Spirit LM: Open Source Multimodal Language Model for Text and Speech
By
–
Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech.
— AI at Meta (@AIatMeta) 18 octobre 2024
Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches… pic.twitter.com/gMpTQVq0nEToday we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches
-
Meta FAIR Releases Eight New AI Research Artifacts for Open Science
By
–
Open science is how we continue to push technology forward and today at Meta FAIR we’re sharing eight new AI research artifacts including new models, datasets and code to inspire innovation in the community. More in the video from @jpineau1.
— AI at Meta (@AIatMeta) 18 octobre 2024
This work is another important step… pic.twitter.com/o43wgMZmzKOpen science is how we continue to push technology forward and today at Meta FAIR we’re sharing eight new AI research artifacts including new models, datasets and code to inspire innovation in the community. More in the video from @jpineau1
. This work is another important step