The obvious interpretation of "train on weak signal, get much stronger behavior" is that the system acquired an internal sycophancy preference which it then went hard on. Do you have any way to check whether that's what happened?
MACHINE LEARNING
-
Extreme PyTorch: ML Workloads and AI Agents Challenges
By
–
writing out my Keynote at @MLSysConf this year and I'm pretty excited!
"Extreme PyTorch: Inside the Most Demanding ML Workloads—and the Open Challenges in Building AI Agents to Democratize Them" -
Reinforcement Learning Enables LLM Reasoning with Single Example
By
–
Reinforcement Learning for Reasoning in Large Language Models with One Training Example Wang et al.: https://
arxiv.org/abs/2504.20571 #ArtificialIntelligence #DeepLearning #MachineLearning -
Reinforcement Learning Enables LLM Reasoning with Single Training Example
By
–
Reinforcement Learning for Reasoning in Large Language Models with One Training Example Wang et al.: https://
arxiv.org/abs/2504.20571 #ArtificialIntelligence #DeepLearning #MachineLearning -
AI at the Crossroads: GPT-4 and the Future of Human-Like Intelligence
By
–
New on the Blog: AI at a Crossroads: GPT-4, The Turing Test, and the Race for Human-Like Intelligence Stay ahead in the ever-evolving world of Digital Transformation. This article explores key insights into emerging technologies and their impact on today's business
-
Model Pretraining and Post-Training Engineering Role
By
–
The job will be related with model pretraining (of existing and new arch), post training and some downstream tasks, other than merely doing some fancy paper oriented research.
-
Epigenetic Age Predicts Dementia Risk Using AI Analysis
By
–
Epigenetic (methylation, biological) age is associated with increased risk of dementia and brain structural changes (including atrophy). From UK Biobank's 280,000 participants, ~4,800 cases of dementia over 14-year follow-up https://
neurology.org/doi/10.1212/WN
L.0000000000213616
… -
AI’s First Decade: Methods Over Benchmarks, RL’s Missing Piece
By
–
Beautifully written post from Shunyu connects the dots the past 10 years in AI:
– Winners of the “first half” of the AI game have been methods papers, not benchmarks
– It turned out the missing piece in getting RL to work was priors (natural language reasoning), obtained in a way -
David Autor discusses technology and jobs impact
By
–
I can't wait to see my friend and old MIT colleague @davidautor next Monday. His research about technology and jobs is always insightful. Please join us!
