Excited to be (finally) heading to the PyTorch Conference! I’ll be giving a talk tomorrow at 11:00 AM on “The LLM Landscape 2025”, where I’ll discuss the key components behind this year’s most prominent open-weight LLMs, and highlight a few architectural developments that go
RESEARCH
-

Compounding AI Predictors: Nine-Point IQ Boost Per Generation
By
–
Incredible. I assume this will eventually compound with better predictors. Imagine if each generation boosts expected IQ by an additional 9 points?
-
Learning Error Recovery Without Creating Errors in Testing
By
–
That’s almost separate… You certainly want to learn recoveries in the event of errors at test time. You just don’t want to also learn to actively create errors.
-
GPT-10 Input Architecture: Pixels Over Tokenization
By
–
Haha. I am afraid people interpreted my “delete tokenizer” as “use bytes directly without BPE”, the issue is you *still* need bytes encoding arbitrariness even for that! Pixels is the only way. Just like humans. It is written. If GPT-10 uses utf8 at the input I will eat a shoe.
-
Process Supervision and Error Recovery in RL Training
By
–
Fair but it’s still actively learning to make those errors (just so it can recover from them later), which imo is still a bit weird. In an ideal world you wouldn’t, possibly process supervision is one way to get there even within RL framework. Def agree on “recovery learning”.
-
Stanford Study: AI Companies Using User Conversations for Training
By
–
A @Stanford study reveals that leading AI companies are pulling user conversations for training. Should users of AI chatbots worry about their privacy?
-
Major AI Conferences Expand Partnership Beyond ICLR 2025 Pilot
By
–
This builds on our successful pilot with ICLR 2025, extending it to all 3 major conferences. We thank the board members of NeurIPS, ICML and ICLR for their help in arranging this partnership!
-
TMLR Papers Eligible for NeurIPS ICML ICLR Presentation
By
–
We at TMLR are proud to announce that selected papers will now be eligible for an opportunity to present at the joint NeurIPS/ICML/ICLR Journal-to-Conference (J2C) Track:
-
J2C Certification: New Prestigious Recognition for TMLR Papers
By
–
To be eligible, papers must receive one of the following certifications: Featured, Outstanding or the new J2C Certification.
We expect J2C Certifications to be a selective and prestigious recognition, with only about 10% of accepted TMLR papers to receive it. -
Data Mixing Balance: Vision-Language Skills and Regional Imbalance
By
–
That's a great question and will take the opportunity to dash off a bit on the data mixing. Mixing data is a tricky balance it turns out. There were two main factors at play: – we wanted to keep general vision-language skills.
– and had unbalanced regions and languages: think
