Agree with you. Do you mean the 32B distilled models or the 3B one I hinted at at the end? I think 32B should be just fine but yeah, 3B will be more tricky; I'd say that one is more for educational purposes.
@rasbt
-
Understanding Reasoning LLMs: DeepSeek R1 and Building Methods
By
–
I just finished writing up my take on reasoning models: https://
magazine.sebastianraschka.com/p/understandin
g-reasoning-llms
…
Here, I
1. Discuss the advantages &d disadvantages of reasoning models
2. Of course, describe and discuss DeepSeek R1
3. Describe the 4 main ways to building & improving reasoning models -
LLM Distillation Term Usage: R1 Dataset Curation and Model Training
By
–
Today, in LLM contexts the term "distillation" is used quite loosely. In the case of R1 it just means that they created and curated a dataset for SFT from R1 that they used to train distilled R1 models based on Qwen and Llama.
-
Questioning the necessity of LLM-based operating systems
By
–
Except, I don't know if we want/need an LLM-based OS . Reminds me a bit of the web 3.0 on blockchain topic
-
GRPO motivation: computational and memory efficiency clarification
By
–
Awesome, thanks a lot for this direct head-to-head comparison! Actually, after reading the R1 paper, wasn't the pure motivation behind GRPO computational (/memory) efficiency?
-
Adding Official OpenAI Transformer Weights to Notebook
By
–
Ah nice! I thought they only had the transformer-library-formatted weights on the hub! In that case, I can actually add that to the notebook directly. Cool stuff! (Btw the reason why I opted for the OpenAI Tf ones was that they were the original/official ones)
-
Lower-precision formats and distilled models enabling local long-context LLMs
By
–
Me neither :). I think with new lower-precision formats, and better distilled models, we'll maybe increasingly adopt long-context LLMs run locally.
-
Long-context needle-in-the-haystack improvements driven by cost optimization
By
–
I think that's exactly what's happening but for cost reasons rather than accuracy reasons. I think long-context needle-in-the-haystack issues have improved a lot last year since all major LLMs now have a dedicated long-context finetuning stage in pre/post-training.
-
Loading weights from Hugging Face model hub as alternative
By
–
Sorry about the hassle loading the weights directly. But I actually have some bonus material on that to load them from the HF model hub as an alternative (should be referenced in the book itself):
-
Pre-training remains superior to expensive knowledge injection methods
By
–
Interesting! But knowledge is still best instilled via pre-training. "Relatively" cheap still remains very expensive and cumbersome, even for 7B models, compared to just adding some more embeddings to a database.