Glad to hear that you've been enjoying it! Regarding RoPE and GQA, I've added some bonus material here a few months back where I implement it from scratch in the GPT -> Llama conversion guide: https://
github.com/rasbt/LLMs-fro
m-scratch/tree/main/ch05/07_gpt_to_llama
…
I was planning to add more, but unfortunately I am currently
OPEN SOURCE
-
RoPE and GQA Implementation in GPT to Llama Conversion Guide
By
–
-
Final Chapter and Bonus Content Highlights for Readers
By
–
Hope that was fun (personally, writing the last chapter was my fav). Hope the bonus content came in handy as well! If you had to prioritize, I'd check out the DPO one and the GPT to Llama conversion guide!
-
Open Source Winning Through Distribution Not Model Weights
By
–
open source is already winning – the most is not in weights, but distribution
-
SemiKong: First Open Source Semiconductor-Focused LLM Model
By
–
SemiKong, a model built with Llama, is the world's first open source semiconductor-focused LLM. With this work @aitomatic is enabling semiconductor companies to build Domain-Expert Agents to capture and scale their deep domain expertise https://
go.fb.me/9yq4mq -
DeepSeek-V3 Technical Paper Released
By
–
read the paper https://
github.com/deepseek-ai/De
epSeek-V3/blob/main/DeepSeek_V3.pdf
… -
Llamathon Hackathon: Win Credits and Dev Tiers
By
–
Get ready to run the Llamathon! 🦙
— SambaNova (@SambaNovaAI) 26 décembre 2024
Join our Llama-thon #Hackathon with @Gradio and get a chance to win:
🤗 $10k in @HuggingFace credits
🚀 Free #dev tiers on SambaNova
… and more!
See details below 👇#AIGet ready to run the Llamathon! Join our Llama-thon #Hackathon with @Gradio and get a chance to win: $10k in @HuggingFace credits Free #dev tiers on SambaNova … and more! See details below #AI
-
Open Source vs Closed Source: Western AI Labs Implications
By
–
you mean this? I was hoping for a bit more Balaji style deep dive and thoughts on what it means for the western closed source labs.
-
Hugging Face collaborates with major Chinese AI labs on open science
By
–
FWIW – we work quite closely with all major Chinese labs, we don't distinguish between any. Wherever there's open source/ science, HF is there.
-
DeepSeek’s 5.5M USD Breakthrough: Frontier AI Cost Revolution
By
–
Agreed. On a similar note, what are your thoughts on the recent DeepSeek release? The frontier *just* costs 5.5M USD (+ a cracked team) – and I'm here for it!
-
Text Generation Inference v3.0.1 release fixes reported issue
By
–
this should not happen, can you try with the latest tagged image – 3.0.1 happy to flag it to the team if it still doesn't work! sorry for the inconvenience! https://
github.com/huggingface/te
xt-generation-inference/releases/tag/v3.0.1
…