Distillation has been on the news (!) due to @deepseek_ai
. The paper https://
arxiv.org/abs/1503.02531 was actually rejected from NeurIPS 2014 due to lack of novelty (true-ish), and lack of impact . Thanks reviewer#2 (literally), and thanks for @arxiv
! @geoffreyhinton @JeffDean

→ View original post on X — @oriolvinyalsml