“Interleaved Head Attention” A core limitation of transformers is that standard attention gives you H isolated heads, which means you only get H independent attention patterns. So this paper lets heads mix before attention by creating pseudo-heads from learned combinations of
@askalphaxiv
-
PaperOrchestra: Multi-Agent Framework Automates AI Research Paper Writing
By
–
"PaperOrchestra: A Multi-Agent Framework for Automated AI Research Paper Writing" This paper builds a multi-agent system for AI research writing. Each part of it handles one job: one plans, one does literature search, one makes figures, one writes, and one revises. This treats
-
ByteDance In-Place Test-Time Training Updates Model Weights
By
–
New Test-Time Training/Continual Learning technique from ByteDance Seed "In-Place Test-Time Training" It lets the model update part of its own MLP weights during inference to store useful information from the current prompt. With no new exotic architecture, it simply just
-
Meta Neural Computer: Model as Integrated Computing System
By
–
What if the model didn’t just use a computer, but actually was the computer? Meta AI introduces "Neural Computer", a model where computation, memory, and I/O are all inside one learned system. Their early prototype learns from screen recordings of terminals and desktops, and it
-
TriAttention: Efficient Long Reasoning with Trigonometric KV Compression
By
–
“TriAttention: Efficient Long Reasoning with Trigonometric KV Compression”
— alphaXiv (@askalphaxiv) 8 avril 2026
Most KV-cache compression methods guess what to keep by looking at recent attention.
But this paper argues that the signal is unstable because RoPE keeps rotating queries with position, so what looks… pic.twitter.com/GrEYc0gZ2b“TriAttention: Efficient Long Reasoning with Trigonometric KV Compression” Most KV-cache compression methods guess what to keep by looking at recent attention. But this paper argues that the signal is unstable because RoPE keeps rotating queries with position, so what looks unimportant now may matter later. So they proposed TriAttention, which looks in the pre-RoPE space and finds that many heads have stable Q/K centers. That lets it predict which token distances a head is likely to retrieve, and compress the KV cache using that structure rather than noisy recent attention. This shift from "keeping what was attended recently” to “keeping what this head is likely to need later” Empirically, it matches full attention on AIME25 with 2.5x higher throughput or 10.7x less KV memory.
-
RLSD: Self-Distilled Reasoning RL with Token-Level Credit Assignment
By
–
“Self-Distilled RLVR”
— alphaXiv (@askalphaxiv) 8 avril 2026
Most reasoning RL rewards are reliable, but too sparse.
Self-Distillation (SD) can fix that with dense token-level signals, but if the teacher sees hidden info, the model can start learning shortcuts it will never have at test time.
So this paper, RLSD,… pic.twitter.com/qHzDTF4wef“Self-Distilled RLVR” Most reasoning RL rewards are reliable, but too sparse. Self-Distillation (SD) can fix that with dense token-level signals, but if the teacher sees hidden info, the model can start learning shortcuts it will never have at test time. So this paper, RLSD, let RL decide whether an answer was good or bad, and let self-distillation decide which tokens deserve more credit. And instead of using a teacher to tell the model what to imitate, they use it to do token-level credit assignment, which gives denser learning than vanilla RLVR, without the instability and leakage of naive self-distillation. Empirically, RLSD stays stable while on-policy SD degrades, and beats GRPO-style baselines on multimodal reasoning.
-
Gemini 3 Flash and GPT-5.4 mini now available
By
–
Now available in addition to Gemini 3 Flash and GPT-5.4 mini. Check out alphaxiv.org!
-
GLM-5.1 Advanced Research Paper Analysis with AI Understanding
By
–
Introducing GLM-5.1 for understanding research papers 🚀
— alphaXiv (@askalphaxiv) 7 avril 2026
Highlight any section of a paper to ask questions and “@” other papers for quick context, comparisons, and benchmark references pic.twitter.com/N0tlqKZuShIntroducing GLM-5.1 for understanding research papers 🚀 Highlight any section of a paper to ask questions and “@” other papers for quick context, comparisons, and benchmark references