Today, in LLM contexts the term "distillation" is used quite loosely. In the case of R1 it just means that they created and curated a dataset for SFT from R1 that they used to train distilled R1 models based on Qwen and Llama.

→ View original post on X — @rasbt,