These papers examine the capabilities of efficient models, including Sparse Transformer, Linear Transformer, and Mamba, revealing significant gaps in reasoning tasks compared to standard Transformers.

→ View original post on X — @jiqizhixin,