And if you are looking for inference mode quantization, you can use `–quantize "bnb.nf4" with the "generate/base.py" scripts as well.
I am currently using that for running CodeLlama 34B models.
LLMS
-
Inference Mode Quantization with BNB NF4 for CodeLlama
By
–
-
QLoRA-style Quantization Added with BNB NF4 Support
By
–
Yes! Just added QLoRA-style quantization the other week, too. You can use it via –quantize "bnb.nf4" for example. I have some benchmarks here:
-
Fine-tune Models to Replace GPT-4 Using LangSmith Data
By
–
This is really useful for finetuning models to replace GPT-4 at particular points in your system Log all data flowing through your app to LangSmith Export with some filter (all runs, all runs with positive feedback, etc) to a dataset Finetune a model on this data!
-
Llama2 Powers Fun Application on Anyscale Compute
By
–
This is very fun! And – really cool that this is powered by Llama2 on @anyscalecompute
-
Cohere and Anthropic sought AWS partnership for computing resources
By
–
Lots of fresh details in this
@theinformation
piece. “In 2021, Cohere and Anthropic had conversations with AWS about an OpenAI-like partnership. The startups wanted AWS to provide access to cloud computing resources and make equity investments…” -
ChatGPT Pro: Master AI Forecasting and Pattern Prediction
By
–
Predict the Future, Literally: Join the league of AI forecasters who turn patterns into predictions. AI expertise can make you a modern-day Guru! This is How to Become a ChatGPT Pro https://
shorturl.at/ajF49 #AI #Innovation #tech #IoT -
Cerebras MBZUAI Develop Arabic-Optimized Language Model Techniques
By
–
(2/2) Researchers at Cerebras, Inception, and MBZUAI developed techniques to:
-Create a custom tokenization scheme optimized for Arabic -Identify the proper mix of English, Arabic, and code data -Augment Arabic tokens with machine-translations Models: https://
huggingface.co/inception-mbzu
ai
… -
Jais: Top-Performing Open-Source Arabic LLM Model
By
–
(1/2) Jais is the top-performing open-source Arabic LLM in the world. Better than Bloom, Llama, Falcon, etc. It is a 13B parameter model that was trained on a brand-new multilingual dataset. Read the technical paper: https://
inceptioniai.org/jais/docs/Tech
nicalpaper.pdf
… -
Remote AI Careers: Global Opportunities in Machine Learning and Robotics
By
–
Remote-Friendly AI Careers? Break geographical barriers and tap into global #AI projects. Whether you're into #machinelearning, #NLP, or #robotics, the digital world is your oyster! #RemoteWork #AICareer #GenAI #IoT
-
Asking ChatGPT to evaluate its output and suggest missing input
By
–
What additional input do you need from me to help you write better output? This basically asks ChatGPT to evaluate its own output, as well as the initial prompt, and suggest additional information to improve the output.