Self distillation isn’t always the silver bullet people think it is We wrote a wiki on self-distillation, informed by several interviews with authors and practitioners. It covers how self-distillation works, when it fails, and other directions like Pedagogical RL and Le