1/Meta just released Llama3.1 405B! @scale_AI partnered deeply with @Meta on this release: SEAL Evaluations: Based on our evals on IF on Math #4 on Coding Enterprise partnership for custom Llama models Data Foundry partnership on RLHF & SFT
RESEARCH
-
Llama 3.1 405B Achieves Top Performance on SEAL Leaderboard
By
–
We evaluated Llama3.1 405B Instruct on SEAL Leaderboard. As a reminder, our evals are:
PRIVATE (no overfitting)
EXPERT EVALUATED (trustworthy)
EVOLVE (no saturation) Our results show Llama3.1 is top notch:
Instruction Following
Math
#4 Coding -
Llama 3.1 Research Paper Deep Dive on LLM Training
By
–
Among the most impressive aspect of the Llama 3.1 release is the accompanying research paper! Close to 100 pages of deep knowledge-sharing on LLMs like we havn't seen very often recently What a treat! It covers everything, pretrainining data, filtering, annealing, synthetic
-
Scaling Machine Learning: Multi-Terabyte Dataset Lessons
By
–
Lessons Learned from Scaling to Multi-Terabyte Datasets https://
bit.ly/4cVZmws
#AI #MachineLearning #DeepLearning #LLMs #DataScience -
Llama License Update: Developers Can Now Use Model Outputs for Improvement
By
–
We’ve also updated our license to allow developers to use the outputs from Llama models — including 405B — to improve other models for the first time. We’re excited about how this will enable new advancements in the field through synthetic data generation and model distillation
-
Llama 3.1 405B Competes with GPT-4 and Claude
By
–
With Llama 3.1, we evaluated performance on >150 benchmark datasets spanning a wide range of languages — in addition to extensive human evaluations in real-world scenarios. These results show that the 405B competes with leading closed source models like GPT-4, Claude 2 and Gemini
-
Llama 3.1 405B: Training the Largest Model at Scale
By
–
Training a model as large and capable as Llama 3.1 405B was no simple task. The model was trained on over 15 trillion tokens over the course of several months requiring over 16K @NVIDIA H100 GPUs — making it the first Llama model ever trained at this scale. We also used the 405B
-
Fine-tuning Llama-3.1-8B with High-Quality Reasoning Datasets
By
–
What I'm excited about: Use Llama-3.1-405B to generate a high quality reasoning dataset, then fine-tune Llama-3.1-8B models.
— Jiquan Ngiam (@JiquanNgiam) 23 juillet 2024
Imagine having 4o-mini performance models, but open sourced, and tuned for your use cases. pic.twitter.com/mf9tMZ8vWMWhat I'm excited about: Use Llama-3.1-405B to generate a high quality reasoning dataset, then fine-tune Llama-3.1-8B models. Imagine having 4o-mini performance models, but open sourced, and tuned for your use cases.
-
AI Conferences Moving to Africa and Latin America
By
–
So there’s no doubt as to why I posted this, my position is I have taught at 2 @Khipu_AI and 3 @DeepIndaba and I raised money for the first 2 Indabas. I highly recommend these inspiring meetings. I strongly campaigned for AI conferences to move to Africa and LatAm. I believe
-
Meta Releases New Llama Model Update Blog Post
By
–
Os dejo el link con el blog post de Meta. Yo voy a repasar la info y más que hacer un hilo, luego abrimos directo y comentamos la importancia de esta noticia 🙂 https://
llama.meta.com