New paper advised by Yann LeCun! "H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning" Most world models plan in a single latent space and at one timescale, which makes long-horizon planning expensive and forces one representation to handle both
@askalphaxiv
-
LiFT: Loop Flow Transformers for Looped Transformer Training
By
–
"LiFT: Loop Flow Transformers" Running more loops than they were trained on usually hurts generation quality for looped transformers. This paper LiFT fixes this by training each loop to progressively correct the model’s initial velocity estimate toward the flow-matching target,
-
Sphere Encoder 2 fixes blurry one-step autoencoder generation
By
–
"Sphere Encoder 2" This paper fixes why one-step autoencoder generation looks blurry. It trains the latent rotation all the way from an encoded image to the 90° equator region where random generation actually starts. Then instead of forcing those highly ambiguous latents to
-
Triadic Linear Attention: 3D Recurrent States for Long-Context Modeling
By
–
"Triadic Linear Attention: Three-Dimensional Recurrent States for Long-Context Sequence Modeling" Linear attention is fast, but its fixed-size state limits memory. This paper expands that matrix into a 3D tensor using two keys, giving an 8x larger recurrent state with only
-
Paper: MCMC sampling improves SFT fine-tuning via data distribution
By
–
"Finetuning with Sampling: SFT Learns Better Than You Think" SFT may be better than we think, and the problem could have just been the data distribution. This paper uses MCMC to rewrite expert traces into equally correct answers that look more like what the base model would
-
Paper: Context Language Models Let Agents Rewrite Their Own Context
By
–
"Context Language Models" Most agents rely on an external harness to manage context. This paper instead lets the model directly rewrite its own context as a file, learning when to compress, delete, preserve, or reorganize information. At zero-shot, it gets 11.4% higher
-
Looped Diffusion Transformer reuses blocks per denoising step
By
–
"Looped Diffusion Transformer" This new paper reuses the same Transformer blocks multiple times within each denoising step, repeatedly updating the hidden state to improve the prediction without increasing the model’s parameter count at inference-time. The found that a 260M
-
SkillRefiner: Agents Improve Skills from Historical Traces
By
–
Can agents improve their own skills without generating any new rollouts? This paper introduces SkillRefiner, which learns entirely from historical agent traces. It turns past mistake patterns into targeted edits to the agent’s existing skill, by basically treat deployment
-
Decoding Looped Transformers Better for Almost Free
By
–
"Decoding Looped Transformers Better for (Almost) Free" This paper treats an early loop from a looped transformer as a rough draft and the final loop as the refined answer. It then uses the difference between them to push the final prediction further in the direction the model
-
Talk on Human-AI Collaboration at Scale study
By
–
Excited to have Yijia Shao giving a talk on her latest work "Human-AI Collaboration at scale" Tomorrow (10/02) 10 am PST! This study investigates how people actually work with AI using the Anthropic Insights system, focusing on task criticality, human agency, learning, and