The just release 30B model from MosaicML looks really great! Nice (big) size, OSS apache-2 licence and long context! take a look at the thread for more details
@thom_wolf
-
OpenAI’s Rapid Execution in AI Model and Product Shipping
By
–
moats or no-moats on gpt4 it's still super impressive how OpenAI is shipping new models, products and ideas so fast and consistently. amazing execution
-
High Quality Data Distillation from Large Noisy Corpora
By
–
More the reverse I guess: current trend is to distill high quality data from large (noisy) corpora
-
Using LLMs to Annotate and Improve Training Data Quality
By
–
This is really really cool! I totally buy the idea of using your current LLM to annotate your full training data and increase the quality of subsequent pretrainings. Training data is the most valuable asset, gives back 1000x the care and love you give it!
-
Text-Generation-Inference: HuggingFace’s Recommended LM Generation Tool
By
–
You should use text-generation-inference, that’s what we’re using at HuggingFace. Don’t use the transformers library for generation with large LM. https://
github.com/huggingface/te
xt-generation-inference
… -
Text-Generation-Inference: HuggingFace’s LLM Inference Solution
By
–
You should try text-generation-inference, that’s what we’re using at HuggingFace. Don’t use the transformers library for generation with large LM. https://
github.com/huggingface/te
xt-generation-inference
… -
Falcon vs LLaMa: Evaluation methodology comparison issues
By
–
Cool though at the moment the comparison with Falcon doesn’t make any sense since the numbers come from two different evaluation setups (EleutherAI harness for Falcon on the leaderboard and Francis setup for LLaMa in this thread).
-
Meta’s MusicGen Model Impresses with Advanced Generative AI Capabilities
By
–
This new MusicGen model from Meta is seriously impressive!
-
Internal RLHF Tool Scaling to Production Open Source
By
–
Interesting that this small Spaces built on the side by our rlhf team and that run on the spare cluster time is becoming a big tool. We should move it to a more stable release with OSS maintainers
-
MMLU Evaluation Benchmarks Updated with HELM Prompts
By
–
Great, thanks for sharing Yao. As mentioned on the leaderboard we are using the MMLU of the Eleuther harness for now but we're in the process of adding all the evals of the MMLU of HELM which is closer to the original prompts soon (actually working on it as we speak this weekd).