What hyperparameters changed when optimizing for muon? @clashluke has tried several new optimizers on our code and not reliably beaten Adamw.
Hyperparameter Changes for Muon Optimization and Optimizer Comparison
By
–
By
–
What hyperparameters changed when optimizing for muon? @clashluke has tried several new optimizers on our code and not reliably beaten Adamw.