Nobody's been talking about it but it's rather *mind-blowing* imo that the open-source Flacon 40B model is topping LLaMa 65B on leaderboards and many evals while having required not even half the compute of LLaMa to train from scratch Quick back of the envelop calculations:
–
@thom_wolf
-
Falcon 40B Outperforms LLaMa 65B With Half Training Compute
By
–
-
Training Falcon Model Variants with Extended Context Windows
By
–
If you have some (big) GPUs laying around there’re a lot of cool variant to train on top of Falcon btw. A longer context would be awesome and of course instruction finetuned (TII original IFT one is already #1 on the open leaderboard). Excited to see what you’ll share!
-
Falcon-40b Language Capabilities and Supported Languages Overview
By
–
It’s not trained on Arabic though. Covers English, German, Spanish, French (and limited capabilities in Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish) from the model card https://
huggingface.co/tiiuae/falcon-
40b
… -
Falcon 40B Now Apache-2 Licensed Free Commercial Use
By
–
The license of the Falcon 40B model has just been changed to… Apache-2 which means that this model is now free for any usage including commercial use (and same for the 7B)
-
Improving LLM Leaderboard Evaluation Metrics and Prompts
By
–
It's a very good point David, also happening in the discussion section of the leaderboard: https://
huggingface.co/spaces/Hugging
FaceH4/open_llm_leaderboard/discussions/26
… Feel free to join the discussion here Maybe we should update the leaderboard with other prompts/evals? -
LLaMa Evaluation Discrepancy: ElutherAI Harness Results Analysis
By
–
Here is a good dive in on LLaMa underperforming on the EleutherAI harness versus the published number (TLDR is that we don't know yet which prompt they used for evaluation):
-
Falcon 40B Dethrones LLaMa on Open Leaderboard
By
–
LLaMa is dethroned A brand new LLM is topping the Open Leaderboard: Falcon 40B *interesting* specs:
– tuned for efficient inference
– licence similar to Unity allowing commercial use – strong performances
– high-quality dataset also released Check the authors' thread https://
x.com/slippylolo/sta
/slippylolo/status/1662082035744227330
… -
Open LLM Leaderboard promotes transparency and innovation in AI
By
–
just saw the shout out to the Open LLM Leaderboard from @Jason in the latest @theallinpod podcast E129 https://
youtube.com/watch?v=C762HW
Sz67w&t=972s
… on how open-source and transparency are super strong catalysts to foster innovation and trust -
Open LLM Evaluation: 228 Community Models in Queue
By
–
Super nice to see so much excitation for open evaluation of large language model. Currently 228 community-submitted models in the queue for being evaluated on the Hugging Face cluster's spare cycles and added to the leaderboard! Check it out here: https://
huggingface.co/spaces/Hugging
FaceH4/open_llm_leaderboard
…