What if you could generate a full training dataset from 1 sentence and 0 seed examples? This new paper from @adaption_ai proposes Invent-a-Dataset, a specialized system that turns a dataset description directly into training data, rather than repeatedly prompting a
@askalphaxiv
-
Alignment Forecasting Predicts Misalignment From Training Data
By
–
"Alignment Forecasting: Predicting Misalignment From Training Data" This paper has an AI inspect the training data and forecast whether fine-tuning will increase behaviors like deception, sycophancy, or sabotage. Their decomposed forecaster reaches 0.801 at predicting whether
-
Wiki on self-distillation: how it works, when it fails
By
–
Self distillation isn’t always the silver bullet people think it is We wrote a wiki on self-distillation, informed by several interviews with authors and practitioners. It covers how self-distillation works, when it fails, and other directions like Pedagogical RL and Le
-
ByteDance’s iLLaDA 8B diffusion model rivals autoregressive LMs
By
–
"Improved Large Language Diffusion Models" ByteDance just made bidirectional masked diffusion on-par with autoregessive LM! This paper iLLaDA trains an 8B Transformer from scratch on 12T tokens, then keeps the same denoising objective for SFT on a 25B-token instruction corpus.
-
Sakana AI’s Fugu dynamically routes queries to multiple models
By
–
Sakana Fugu Technical Report Instead of training one larger model, Sakana AI trains an orchestrator that reads each query and dynamically routes or composes GPT-5.5, Gemini-3.1-Pro, Claude Opus 4.8 and other agents into query-specific workflows. With Fugu being the fast router,
-
PACE: Training optimizers for the averaged model you return
By
–
"Training for the Model You Return" Most LM pipelines return an EMA or averaged checkpoint, but optimizers still train like the final iterate is what matters. So if the output is an averaged model, can we shape training so that average gets better? This paper introduces PACE,
-
Autodata: An agentic data scientist to create high quality synthetic data
By
–
"Autodata: An agentic data scientist to create high quality synthetic data" If there's auto-research, shouldn't there also be a auto-data generation? In this new Meta paper, they proposed Autodata, which makes synthetic data generation work more like a data scientist, with an
-
Tapered LMs: Early layers write more, later layers refine
By
–
“Tapered Language Models” Most LMs give every layer the same MLP width, but the paper shows this is probably wasteful. Early layers seem to write more new information into the residual stream, while later layers mostly refine what is already there. So instead of making the
-
GLM 5.2 enables trusted open model for automated research tasks
By
–
1/2 This is obviously not a comprehensive benchmark, but it’s clear that we finally have an open model that can be trusted and depended upon on difficult research tasks. You can easily run autoresearch yourself with GLM 5.2 by changing ‘arxiv’ to ‘autoarxiv’ for any arXiv URL:
-
Comparison of GLM 5.2 and Opus 4.8 on SDPO paper reproduction
By
–
Here’s a fun comparison between GLM 5.2 and Opus 4.8 on a one-shot reproduction of the SDPO paper This is a hard task: the model must resolve messy verl issues and then run ablations to completion and confirm the paper’s claims. – GLM 5.2 costs $6.21 while Opus 4.8 cost us