Just implemented LoRA (
https://
github.com/Lightning-AI/l
it-parrot/pull/128
…) and may be able to tell more soon. (Stay tuned for more info next week. Have to get all my analyses organized and write that up properly before mixing up some numbers there. Probably no qlora, but some info on LoRA at least)
OPEN SOURCE
-
LoRA Implementation in Lit-Parrot with Analysis Coming Soon
By
–
-
Mark Zuckerberg on Meta’s AI Future and Path to AGI
By
–
Here's my conversation with Mark Zuckerberg, his 2nd time on the podcast. We talk about the future of AI at Meta, Facebook, Instagram, and WhatsApp, both near-term open source development of AI and the journey of human civilization toward building AGI. https://
youtube.com/watch?v=Ff4fRg
nuFgQ
… -
RedPajama Open-Source Models: Data Collection & Training Insights
By
–
Next: A fireside chat with Braden Hancock, co-founder @SnorkelAI & Ce Zhang, CTO @TogetherCompute
.They'll discuss the creation of RedPajama, a set of fully open-source models. Uncover the process behind data collection & training. Exciting times for the #OpenSourceAI community -
Replit GitHub Direct Import: One-Click Project Deployment
By
–
3/ Github Direct Import
— Replit ⠕ (@Replit) 8 juin 2023
If your open-source project is meant to run on Replit, you’ll be stoked to hear that developers can import and run it on Replit in one click.
Use https://t.co/OMo25aeUtl to instantly create a Repl from a certain @github repository. pic.twitter.com/ObuWsv30y13/ Github Direct Import If your open-source project is meant to run on Replit, you’ll be stoked to hear that developers can import and run it on Replit in one click. Use http://
replit.com/new/github/use
r/repo
… to instantly create a Repl from a certain @github repository. -
LangChain Release Overview: Key Components for AI
By
–
releases like these are always fun cause you get a full overview of all the langchain components… vectordb, llms, embeddings, document loaders, tools, databases
-
Falcon LLM Demonstrates Open Source AI Community Benefits
By
–
The falcon is soaring! LLM model shows potential of open source AI for the benefit of the wider community
-
Internal RLHF Tool Scaling to Production Open Source
By
–
Interesting that this small Spaces built on the side by our rlhf team and that run on the spare cluster time is becoming a big tool. We should move it to a more stable release with OSS maintainers
-
H2O AI Recognized for Best Custom Generative AI Models
By
–
Big thanks to @eweeknews for recognizing @h2oai
's work in building the "Best Custom Generative AI Models." We're thrilled to be leading the way in #opensource #GenerativeAI products. #LLMs #h2oGPT #h2oLLMStudio Check out the article here: -
Single Epoch Training on 52k Alpaca Dataset
By
–
This was for one iteration/epoch over the 52k Alpaca dataset.
-
LLaMA-Adapter Works with Any LLM, Including Falcon 40B
By
–
Fun fact: LLaMA-Adapter is not LLaMA-specific. You can use it to finetune any LLM.
— Sebastian Raschka (@rasbt) 8 juin 2023
Below, I finetuned a 40B Falcon model using LLaMA-Adapter (it also works on a single GPU with 20 GB RAM).
Have been heads down tinkering & will follow up with more benchmarks! https://t.co/2Fr6faChv2Fun fact: LLaMA-Adapter is not LLaMA-specific. You can use it to finetune any LLM. Below, I finetuned a 40B Falcon model using LLaMA-Adapter (it also works on a single GPU with 20 GB RAM). Have been heads down tinkering & will follow up with more benchmarks!