Ah nice! I thought they only had the transformer-library-formatted weights on the hub! In that case, I can actually add that to the notebook directly. Cool stuff! (Btw the reason why I opted for the OpenAI Tf ones was that they were the original/official ones)
RESEARCH
-
Deep Learning’s Legendary Appetite for Compute Resources
By
–
I don't have too too much to add on top of this earlier post on V3 and I think it applies to R1 too (which is the more recent, thinking equivalent). I will say that Deep Learning has a legendary ravenous appetite for compute, like no other algorithm that has ever been developed
-
Long-context needle-in-the-haystack improvements driven by cost optimization
By
–
I think that's exactly what's happening but for cost reasons rather than accuracy reasons. I think long-context needle-in-the-haystack issues have improved a lot last year since all major LLMs now have a dedicated long-context finetuning stage in pre/post-training.
-

DeepSeek-R1 Training Cost: Missing R1 Distillation Expenses
By
–
The real question: What is the DeepSeek-R1 training cost? The $5.567M DeepSeek cost is missing the cost of training the R1 model to get distilled data. Table 9 in the DeepSeek v3 paper shows that the R1 distillation step is critical for quality. The R1 paper doesn't talk about
-
Pre-training remains superior to expensive knowledge injection methods
By
–
Interesting! But knowledge is still best instilled via pre-training. "Relatively" cheap still remains very expensive and cumbersome, even for 7B models, compared to just adding some more embeddings to a database.
-

2025: The Year of LLM Specialization and Multimodal Models
By
–
Remember when we were arguing about the term "foundation models"? It feels like ages ago! With those base models culminating in DeepSeek v3 in Dec 2024, 2025 will likely be the year of LLM specialization! 1. Multimodal LLMs: It was my big prediction for 2024. Most products
-
DeepSeek’s Cost-Efficient Training Methods Explained
By
–
One explanation of how DeepSeek was able to train its model so cheaply..
-

Free MLops Repository with Talks Books and Papers
By
–
This repository on ML operations has free talks, books, papers and more: https://
bit.ly/MLops Developed by @visenger -
AI Energy Crisis Urgency: Insights from Industry Experts
By
–
Thank you to @MarkVenables6 @illuminaireNews for highlighting @RodrigoLiang
's recent insights on the urgency of the #AI #energy crisis. Read more -
DeepSeek’s 10X Efficiency Breakthrough Reshapes AI Resource Demands
By
–
Take your intelligence estimates for Project Stargate and multiply them by 10. Every AI lab now has the means to make their models 10X+ more efficient because of DeekSeek. This doesn’t mean AI and chips will be used 10X less: the 10X cost decrease means it gets used 10X more.