3/ Abacus Embeddings – achieves 99% accuracy on 100-digit addition problems by training on only 20-digit numbers with a single GPU; the main challenge this work addresses is the inability of transformers to track the exact position of digits.

→ View original post on X — @dair_ai