(8/n) Using dense optimal HPs, the SP and μP models experience detuning HPs as sparsity increases. This detuning is a confounding variable present in many existing sparse studies. SμPar eliminates this confounding variable and enables more robust sparsity studies.

→ View original post on X — @cerebras