Adam-atan2 is a neat one line code change to remove the epsilon hyperparamter from Adam by changing the divide to an atan2(). This feels like a generally useful trick for many places you might be worried about divide by zero and numeric precision issues.

→ View original post on X — @id_aa_carmack