1/4 We hit a strange logprob mismatch while training Jamba 3B with GRPO. Rollout logprobs and training-side recompute should match before any weight update. Ours didn't. That was the canary. 🧵 [Translated from EN to English]
Discovery of a logprob anomaly during Jamba training
By
–
