The gap between closed-source and open-weight models is closing in terms of MMLU The release of GPT-4o is not a significant breakthrough with a score of 87.2% on the 5-shot MMLU (source: https://
openai.com/index/hello-gp
t-4o/
…). On the other side, Meta reported that Llama 3 405B's
@maximelabonne
-
Llama 3 405B Closes Gap with GPT-4o on MMLU
By
–
-
LLM Course Reaches 30k Stars on GitHub
By
–
The LLM Course hit 30k stars on GitHub! This is beyond anything I imagined. To compare, vLLM has 20k stars and Jax has 28k. We're not at llama.cpp (58k) or PyTorch (78k) yet, but this already feels indecent. Thanks for all the support! I'll be adding new sections and
-
Quarto Publishing Platform for Technical Blogging
By
–
Just updated my blog with new info, links, and various fixes: https://
mlabonne.github.io/blog/ Once again, this is the time to recommend @quarto_pub if you want to start a blog. It natively handles markdown and Jupyter notebooks. It's customizable, easy to use, and looks pretty good! -
Improving AI Benchmarks: TIGER-Lab’s Enhanced MMLU Evaluation
By
–
We need to improve our benchmarks the same way we improve our models. Super interesting work from TIGER-Lab with an upgraded version of MMLU with 12k complex questions (vs. 16k for MMLU) and additional reasoning problems. – It is more discriminative among frontier models (see
-
GPT-4o Widens Lead in LMSYS Chatbot Arena Competition
By
–
The LLM wars intensify as GPT-4o takes a significant lead – The gap is widening again after @OpenAI
's latest release. Based on data shared by @LiamFedus (
https://
x.com/LiamFedus/stat
us/1790064963966370209
…), GPT-4o significantly outperforms competitors on the LMSYS Chatbot Arena. – @RekaAILabs also -
Joining Google Developer Experts as AI/ML Expert
By
–
Happy to announce that I've joined the @GoogleDevExpert group as an AI/ML expert It's a nice opportunity to travel more, and connect with LLM experts worldwide. Expect more content inspired by my interactions with the LLM community. If you're interested in meeting up at an
-
Hands-On GNN Python Book Becomes Amazon Best-Seller
By
–
I'm proud to announce that the Hands-On #GNN Using #Python book is the best-selling new release in Machine Theory on Amazon! Thanks everyone for your support, your kind emails and DMs.