Today's issue is about the race for open source LLMs that seems to speed up literally every week. This week @theinformation reported that Meta is working on making its LLaMA model commercially permissive. Meta's been evaluating this since as early as May.
OPEN SOURCE
-
DeepSpeed Stage 3 LoRA Bug and FSDP Migration Plans
By
–
Theoretically, you can get it to work if you use deepspeed stage 3 with offloading. But I see there is currently a little bug for stage 3 and LoRA. I think we watned to switch from DeepSpeed to FSDP anyway though. PS: Glad you like my book!
-
Macaw-LLM: Multi-Modal Language Model with Image Audio Video Text
By
–
Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration paper page: https://
huggingface.co/papers/2306.09
093
… Although instruction-tuned large language models (LLMs) have exhibited remarkable capabilities across various NLP tasks, their effectiveness on other data -
WizardCoder: Enhancing Code LLMs with Evol-Instruct
By
–
WizardCoder: Empowering Code Large Language Models with Evol-Instruct paper page: https://
huggingface.co/papers/2306.08
568
… Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely -
h2oGPT: Democratizing Large Language Models
By
–
h2oGPT: Democratizing Large Language Models paper page: https://
huggingface.co/papers/2306.08
161
… Foundation Large Language Models (LLMs) such as GPT-4 represent a revolution in AI due to their real-world applications though natural language processing. However, they also pose many significant -
Agreement on Twitter Algorithm Open Source Release
By
–
Totally agree! https://
github.com/twitter/the-al
gorithm
… -
EnCodec Code Released on GitHub Under MIT License
By
–
The code is available on GitHub for you to build on, released under the MIT license https://
github.com/facebookresear
ch/encodec
… -
AI-Generated Functional QR Codes Model Available on Hugging Face
By
–
AI-generated functional QR codes model on @huggingface
: https://
huggingface.co/DionTimmer/con
trolnet_qrcode
… -
New Content Policy for Open Collaborative Responsible Machine Learning
By
–
We are committed to advancing Open, Collaborative, and Responsible Machine Learning. Introducing our new Content Policy, dedicated to fostering an inclusive and supportive environment for our community. Check it out!
-
QLoRA, Prefix Tuning, and LoRA: Fine-tuning Techniques Comparison
By
–
This was already a lot of coding. I am saving QLoRA for another day! @Yampeleg Regarding prefix tuning/LLaMA-Adapter vs LoRA. The performance is similar, but one advantage of the former is that it allows multimodal inputs (but that's also a post for another day :P)