This is something to remind where it start They all start from there for #LLM or i'm wrong ? https://
pile.eleuther.ai
OPEN SOURCE
-
The Pile: Where Large Language Models Begin
By
–
-
Llama 3 Model Size Increase: 7B to 8B Explanation
By
–
Some context on why our smallest Llama 3 model went from 7B → 8B. More details on the changes to the tokenizer in the full conversation with @astongzhangAZ ➡️ https://t.co/cKuUfwmuZ4 pic.twitter.com/OfqCJtfSJW
— AI at Meta (@AIatMeta) 16 juillet 2024Some context on why our smallest Llama 3 model went from 7B → 8B. More details on the changes to the tokenizer in the full conversation with @astongzhangAZ https://
youtu.be/Tmdk_H2WDj4 -
Mistral Releases Mathstral 7B and Codestral Mamba Models
By
–
Today we are releasing two small models: Mathstral 7B and Codestral Mamba 7B. On the MATH benchmark, Mathstral 7B obtains 56.6% pass@1, outperforming Minerva 540B by more than 20%. Mathstral scores 68.4% on MATH with majority voting@64, and 74.6% using a reward model. Codestral
-
Mistral Releases MathΣtral and Codestral Mamba Research Models
By
–
As the Olympic season reaches Paris, we express our respect to ancient Greeks by releasing two new research models, MathΣtral and Codestral Mamba.
-
Mistral Releases Mathstral and Codestral-Mamba Models
By
–
https://
mistral.ai/news/mathstral/ https://
mistral.ai/news/codestral
-mamba/
… -
Understanding ROS2 Internal Workings and DDS Middleware
By
–
ROS is a common component in robotics, with many technical tutorials and resources available online. However, through this blog, our objective is to provide a detailed understanding of the internal workings of ROS2, how DDS works, the need for DDS, the ROS1 middleware… pic.twitter.com/t2fiAiuYYA
— Satya Mallick (@LearnOpenCV) 16 juillet 2024ROS is a common component in robotics, with many technical tutorials and resources available online. However, through this blog, our objective is to provide a detailed understanding of the internal workings of ROS2, how DDS works, the need for DDS, the ROS1 middleware
-
ZenML integrates HuggingFace dataset viewer for data visualization
By
–
Wow The @zenml_io team integrated the HF embedded dataset viewer inside their app. It looks awesome It's a simple iframe you can drop into any Web app or website to visualize your data. Hat/tip: @htahir111
-
Dolphin 2.9.3-Yi-1.5-34B Fine-Tune Reaches 111k Downloads
By
–
Kudos to @bartowski1182 and @cognitivecompai for the remarkable Yi fine-tune: Dolphin 2.9.3-Yi-1.5-34B-32k-GGUF on @huggingface
! With over 111k downloads last month, your work is a game-changer in the AI community. Check out the model here: -
Rio Yokota Presents Neural Scaling Laws Research at ICML
By
–
Sign up now to attend to hear Rio Yokota present at the 7th workshop on Neural Scaling Laws at ICML 2024 Rio from the Tokyo Institute of Technology will present research on continual pre-training of open-source models on Japanese text. July 22, 2024, 9:00AM Local Time
-
Layer Replication and Pruning Techniques for Model Optimization
By
–
These experiments re-confirm a lot of recipes discovered by doing frankenmerges. – Replicating layers is helpful for creative tasks but not for reasoning (in general) – Changing the layer order doesn't work – Pruning works best with middle layers but still requires healing