MACHINE LEARNING

→ View original post on X — @askalphaxiv

Self-Distillation Fine-Tuning: On-Policy Continual Learning from Expert Demonstrations

By

@askalphaxiv

–

29 January 2026 16h13

2026 is the year of continual learning And we are getting some amazing papers towards that This paper introduces Self-Distillation Fine-Tuning (SDFT): on-policy continual learning from expert demonstrations, with no explicit reward inference or engineering The trick here is:

→ View original post on X — @dair_ai

World Models Replacing Video Generation in 2026

By

@dair_ai

–

29 January 2026 15h30

World models are going to take over in 2026. While video generation models are impressive dreamers, they aren't world simulators. Generative video models can produce stunning clips, but they hallucinate pixel transitions based on statistical correlations. This leads to

Investigation: From Confident Gibberish to Scheduler Fix

By

–

29 January 2026 13h56

5/5 Wrote up the complete investigation-from confident gibberish to the scheduler fix. Includes the false starts (suspected CUDA bugs), the breakthrough (request tracking), and lessons for anyone running stateful models at scale. Read the blog:

GPU Memory Utilization Bug: Token Classification Mismatch

By

–

29 January 2026 13h56

2/5 Used gpu_memory_utilization=0.2 to reproduce quickly, but this happens naturally when the scheduler runs out of token budget and GPU blocks get recycled. New request gets 1 token → misclassified as "decode" But num_computed_tokens=0 → should be "prefill".

SSM Recurrence Security: Mamba State Propagation vs Attention Safety

By

–

29 January 2026 13h56

3/5 SSMs are recurrent, each token depends on previous state. Mamba decode reads state → updates it. Misclassified request reads garbage → propagates recursively through all tokens. Attention writes K/V first, then attends. Safe even if misclassified.

Debugging vLLM: Silent Corruption Bug in Jamba RL Training

By

–

29 January 2026 13h56

1/5 Debugging vLLM: The silent corruption bug
1/1000 Jamba generations collapsed into confident gibberish during RL training. No crashes, no errors, just wrong outputs with high logprobs. The culprit? A scheduler edge case that only triggers under memory pressure.