turns out we're all biased toward familiar text when rating AI outputs. RLHF (Reinforcement Learning from Human Feedback) learned this preference, sharpened it, and now every model collapses into repetition. the fix? ask for probability distributions instead of single answers.
LLMS
-
Mitigating LLM Repetition via Probabilistic Sampling
By
–
LLMs have a repetition problem. ask for a joke → same joke every time ask to roll dice → always returns 4 ask for creative ideas → predictable garbage Try this instead: Generate 5 responses with their corresponding probabilities, sampled at random from the tails of the
-
New research establishes scaling laws for reinforcement learning
By
–
This paper basically gives RL its own “Kaplan moment.” Pretraining had scaling laws. Now RL does too. If this generalizes, we’ll finally move from “try random tricks” → “predict compute curves.” The science of post-training has officially begun. Read the full paper here:
-
Empirical Evidence of Stable Scaling Laws Across Model Architectures
By
–
Here’s the wildest part… The same scaling law held across different model sizes, tasks, and sequence lengths. 8B dense model? 17B×16 MoE? Math + code RL tasks? Even 32k-token reasoning traces stayed on-curve. Stable scaling across everything.
-
Predicting AI Model Performance from Small-Scale Training Runs
By
–
Why this matters: For the first time, we can predict asymptotic performance from small runs. Meta extrapolated 8k GPU-hour curves →
and nailed the actual performance at 100k GPU-hours. That’s predictive scaling in action no more trial and error. -
ScaleRL: Key Components Explained
By
–
ScaleRL isn’t magic. It’s a recipe of small, proven ingredients: – Pipeline-RL async setup
– FP32 logits for stable gradients
– CISPO loss (clipped importance sampling)
– Prompt-level averaging
– Batch-level normalization
– No-Positive-Resampling (drop “too easy” prompts) -
ScaleRL outperforms in RL scaling tests
By
–
They tested this with 400,000 GPU hours across multiple RL recipes: DeepSeek (GRPO)
Qwen (DAPO)
Magistral
Minimax Result: only ScaleRL showed a stable, predictable trajectory. The others broke scaling laws entirely their curves collapsed. -
Meta’s ScaleRL reveals predictable RL scaling laws
By
–
Holy shit… Meta just cracked the art of scaling RL for LLMs. For the first time ever, they showed that "reinforcement learning follows predictable scaling laws" just like pretraining. Their new framework, 'ScaleRL', fits a sigmoid compute-performance curve that can forecast
-
VIR-Bench: Evaluating Multimodal LLMs on Travel Video Understanding
By
–
How well can multimodal LLMs understand long-distance travel videos? Enter VIR-Bench, a new benchmark with 200 real-world travel videos that challenges models to reconstruct itineraries and reason over extended geospatial-temporal trajectories. Why it matters: mastering
-
RiskPO: Risk-Based Policy Optimization for LLM Post-Training
By
–
#PapersAccepted by Jiqizhixin
Our report: https://
mp.weixin.qq.com/s/9TbUIT6ed_wO
viVU0GuLqg
… RiskPO: Risk-based Policy Optimization via Verifiable Reward for LLM Post-Training Peking University
Paper: https://
arxiv.org/abs/2510.00911
v1
…
Code: https://
github.com/RTkenny/RiskPO