Half of this lib is in Rust. Super fast.
@thom_wolf
-
TGI: Fast Production-Grade Open Source LLM Inference Library
By
–
If you find the Falcon too slow out-of-the-box in the transformers library, you should probably play with the super-fast production-grade OSS inference generation library TGI. Check it out here: https://
github.com/huggingface/te
xt-generation-inference
… -
TGI: Fastest Open Source LLM Inference Generation Server
By
–
you should use it in the fastest OSS inference generation server out there: TGI (
https://
github.com/huggingface/te
xt-generation-inference
…) Olivier the maintainer has been optimizing it with all his secret knowledge (well not so secret since it's an open-source repository hahaha) -
Falcon 40B Outperforms LLaMa 65B With Half Training Compute
By
–
Nobody's been talking about it but it's rather *mind-blowing* imo that the open-source Flacon 40B model is topping LLaMa 65B on leaderboards and many evals while having required not even half the compute of LLaMa to train from scratch Quick back of the envelop calculations:
– -
Training Falcon Model Variants with Extended Context Windows
By
–
If you have some (big) GPUs laying around there’re a lot of cool variant to train on top of Falcon btw. A longer context would be awesome and of course instruction finetuned (TII original IFT one is already #1 on the open leaderboard). Excited to see what you’ll share!
-
Falcon-40b Language Capabilities and Supported Languages Overview
By
–
It’s not trained on Arabic though. Covers English, German, Spanish, French (and limited capabilities in Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish) from the model card https://
huggingface.co/tiiuae/falcon-
40b
… -
Falcon 40B Now Apache-2 Licensed Free Commercial Use
By
–
The license of the Falcon 40B model has just been changed to… Apache-2 which means that this model is now free for any usage including commercial use (and same for the 7B)
-
Improving LLM Leaderboard Evaluation Metrics and Prompts
By
–
It's a very good point David, also happening in the discussion section of the leaderboard: https://
huggingface.co/spaces/Hugging
FaceH4/open_llm_leaderboard/discussions/26
… Feel free to join the discussion here Maybe we should update the leaderboard with other prompts/evals? -
LLaMa Evaluation Discrepancy: ElutherAI Harness Results Analysis
By
–
Here is a good dive in on LLaMa underperforming on the EleutherAI harness versus the published number (TLDR is that we don't know yet which prompt they used for evaluation):