read more: https://alphaxiv.org/abs/2603.15031
@askalphaxiv
-
Attention Residuals: Selective Computation in Deeper Transformers
By
–
“Attention Residuals” is now available on AlphaXiv! In standard transformer, every layer just inherits an equal sum of all earlier layers, so as models get deeper, useful computations get diluted instead of being selectively reused. The research team at @Kimi_Moonshot proposes
-
MaxRL: Rethinking Reinforcement Learning Optimization
By
–
RL Isn’t Actually Optimizing What We Think, And That’s a Problem Come join us for this AI4Science talk: Maximum Likelihood Reinforcement Learning (MaxRL). In this session, the author of MaxRL @FahimTajwar10 will cover their paper that takes a step back and asks a fundamental
-
AI-Driven Scientific Discovery Using Bayesian Surprise Methods
By
–
Curious about how AI is starting to do science on its own? Come join us for this AI4Science talk: Data-driven Discovery at Ai2, Open-ended Scientific Discovery via Bayesian Surprise. At Ai2, this line of work has been building toward the idea of using AI to extract genuinely
-
LeCun’s Temporal Straightening for Latent Planning Paper
By
–
Yann LeCun is pumping out papers recently “Temporal Straightening for Latent Planning” This paper shows that by straightening latent trajectories in a world model, Euclidean distance starts to reflect true reachable progress, so it's closer to geodesic/minimum-step distance.
-
Dense Task Experts in Pretrained Neural Networks Thickets
By
–
RL is no longer needed? "Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights" This paper argues that large pretrained models don’t sit at a single optimal set of weights but inside a dense “thicket” of nearby task-specific experts. So once pretraining is
-
Neural Cellular Automata Pre-Training for Language Models
By
–
Pre-Pre-Training on Neural Cellular Automata? This paper argues that LLMs may not need human text to learn core thinking machinery. So by first training on synthetic neural cellular automata, the model learns transferable rule-and-pattern inference that speeds up and improves
-
IndexCache Accelerates Sparse Attention via Cross-Layer Index Reuse
By
–
A really cool paper from @Zai_org "IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse" This paper shows that in sparse-attention LLMs, nearby layers usually pick almost the same important tokens. So you can cache and reuse those token indices instead of
-
Rubric-as-Reward Approach Advances Image Captioning Models
By
–
Really cool rubric-as-reward work for image captioning!