“The Recurrent Transformer: Greater Effective Depth and Efficient Decoding” Transformers are great at parallel processing, but they’re shallow through time, as each layer only lets tokens interact once. This paper changes that by storing keys/values from each layer’s output,
@askalphaxiv
-
Hackathon Success: 140 Participants Implement AI Papers
By
–
Our first-ever hackathon with @marimo_io to implement papers just ended! With over 140 participants and their amazing submissions, we’re excited to announce the winners in an in-person event in SF, and live demos will be presented by both our team and Marimo. A livestream will
-
Abstract Chain-of-Thought: Efficient Latent Reasoning Without Words
By
–
"Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought" Do reasoning models really need to think in words? This paper replaces long verbal CoT with a short learned sequence of abstract tokens that acts like a latent scratchpad. Warmed up from
-
Scientific Theory of Deep Learning Emerges
By
–
“There Will Be a Scientific Theory of Deep Learning” This paper argues that a real scientific theory of deep learning is beginning to emerge. Not a theory that tracks every neuron individually, but a physics-like theory of learning itself. One that aims to characterize how
-
Image Generators as Vision Foundation Models Through Instruction Tuning
By
–
"Image Generators are Generalist Vision Learners" This paper argues that image generation training acts like vision pretraining. They use light instruction-tuning to turn a pretrained image generator into a generalist vision model. With the key idea being representing vision
-
Hyperloop Transformers: Memory-Efficient LLM via Looped Architecture
By
–
"Hyperloop Transformers" This paper propose a memory-efficient LLM via looped Transformers. They basically reuse the middle block across depth, then add hyper-connections only between loops. Key result is that this restores flexibility lost from weight sharing, letting the
-
Token Importance in On-Policy Distillation with Selective Training
By
–
"TIP: Token Importance in On-Policy Distillation" This paper introduces selective token training for on-policy distillation, relying on student entropy to find high-signal tokens. A key point is that entropy misses confident mistakes, so they add teacher-student divergence to
-
Discover the Scientific Article alphaXiv 2604.14084
By
–
read more: https://alphaxiv.org/abs/2604.14084
-
Scaling Self-Play with Self-Guidance for Theorem Proving
By
–
"Scaling Self-Play with Self-Guidance" The main problem with self-play for theorem proving is that generator usually learns to reward hack, which produces messy hard problems that do not help the solver. This paper suggests by adding a Guide model that scores generated