This is the proudest release of my career 🙂 At @liquidai
, we're launching three LLMs (1B, 3B, 40B MoE) with SOTA performance, based on a custom architecture. Minimal memory footprint & efficient inference bring long context tasks to edge devices for the first time!
@maximelabonne
-
Liquid AI Launches Three LLMs with SOTA Performance and Edge Optimization
By
–
-
Developer Celebrates 3K Milestone on Hugging Face Platform
By
–
Haha congrats, I just passed the 3k bar! 🙂 https://
huggingface.co/mlabonne -
Genetic Algorithms and Evaluation Cost Trade-offs in AI
By
–
Yes I've tried that https://
x.com/maximelabonne/
status/1767124527551549860
… (never trust genetic algorithms when the cost of eval is so high however) -
Selecting and Adapting Recipes for Model Implementation
By
–
The safest way is to pick a recipe that already works and transpose it to your use case. That requires using the same source model indeed.
-
Model Merging Techniques and Hidden Representation Dynamics
By
–
I'd say it barely works tbh… 🙂 It really depends on the type of merge and whether you retrain it or not. In this case, it's not really "working". The additional layers seem to push the hidden representations into new areas, which may or may not be interesting, depending on
-
Distillation vs DPO: Model Optimization Techniques Comparison
By
–
Yes, for sure. Distillation would be ideal but DPO would also work.
-
Model Merges and 34B Evaluation Updates
By
–
I would usually assume that but my merges do work though. It might have been fixed after the 34B eval?
-
Self-Merge Experiments: 52B Model Success in BBH and MATH
By
–
I want to share some results related to my self-merge experiments: – The 47B merge doesn't work: replicating blocks > layers (apologies for doubting it) – The 52B is successful in BBH and MATH Lvl 5. However, there might be an issue with the original model's MATH eval. This is
-
Fine-tuning and Model Merging: Practical Tips and Tricks
By
–
Here's my @aiDotEngineer talk in SF about fine-tuning and model merging It's a quick overview of the field with practical tips and tricks to make the best merges and finetunes. Hope you'll like it! 🙂
-
Autocompletion Model and Markdown Compatibility Questions
By
–
Do you know the model they use for autocompletion? Also, does it work with markdown?