Can a new optimizer keep your LLM training stable without sacrificing performance? Researchers at Chinese University of Hong Kong, Max Planck Institute, and Westlake University introduce Pion. Instead of adding adjustments like Adam or Muon, Pion rotates each weight matrix
Pion Optimizer Rotates Weight Matrices for Stable LLM Training
By
–
