i would love for @karpathy to make a nanoGPT for MoEs with multimodal tokenization support
OPEN SOURCE
-
LlamaIndex 101: Unleash LLM Power Over Your Data
By
–
LlamaIndex-101: Unleash the power of LLMs over your data:
-
Model Distillation Playbook for Smaller Cheaper LLMs
By
–
Ready to migrate off of those #expensive oversized commercial #LLMs? Check out our Model #Distillation Playbook containing all of our best practices for turning large models into #smaller, faster, cheaper LLMs! https://
pbase.ai/3OLEBKj -
Building Corrective RAG with Open Source Local LLMs
By
–
Building Corrective RAG from scratch w/ open source and fully local LLMs (Video) Building LLM apps with more complex logical flows can be challenging with smaller, local LLMs. Graphs offer one way to tackle this, laying out the logic flow as a graph and using local LLMs for
-
Open Source AI and Compute Control Essential for Users
By
–
@ghostITCITM very smart approach, 100% agree with you. Ai needs to be open source in our hand as also the compute
-
Mistral-Next LLM Drops: GPT-4 Level Coding Performance Achieved
By
–
Ok, so… today is crazy. There's a mysterious Mistral-Next model that just dropped, and I am testing it for coding—it's GOOD. Possibly GPT-4 level. I even ran a few tests that I asked GPT to evaluate the outputs, and it gave Mistral-Next a perfect score.
-
Mistral Large: A New Open Source Gem Arrives
By
–
Mistral Large on its way! Opensource got a new gem.
-
Text-to-Video AI Joining Generative AI Revolution
By
–
So excited to see text to video joining the party. You can learn more about this specific task here including with some open-source models and datasets: https://
huggingface.co/tasks/text-to-
video
…. AI is going to transform every single modality, domain and tasks of technology building! -
Fine-tuning GPT2 with additional layers avoids pretraining scratch
By
–
he’s pointing out we can just add the two layers I drew here and train those starting from pretrained GPT2 or something; don’t have to pretrain from scratch
-
Smaug-72B Tops Hugging Face Leaderboard as Best Open-Source Model
By
–
Two weeks ago we released Smaug-72B – which topped the Hugging Face LLM leaderboard and it’s the first model with an average score of 80, making it the world’s best open-source foundation model. We applied several techniques on a fine tune derived from a Qwen-72B for this model.