Maybe one day supervised pretraining and SFT will be gone but it’s not clear how. RL has its issues too, and maybe one day we’ll have something smarter.
LLMS
-
Reinforcement Learning Enables Causal Knowledge Beyond Supervised Fine-Tuning
By
–
Thanks for these thoughts. Very helpful. RL does indeed enable us to generalise better. I fully agree. In principle, RL results in causal knowledge whereas SFT only on association. But I feel the question is still relevant because we still rely heavily on (self-)supervised
-

GPT-5 Pro Advances Math With Human Expert Guidance
By
–
We are starting to see some nuanced discussions of what it means to work with advanced AI In this case, GPT-5 Pro was able to do novel math, but only when guided by a math professor (though the paper also noted the speed of advance since GPT-4). The reflection is worth reading.
-
GPT5 Pro Demonstrates Advanced Capabilities in Medical Cardiology
By
–
jeez and GPT5 Pro on healthcare is capable of some advanced cardiology lol
-
Limiting LLM Count Below 20 Becomes Increasingly Difficult
By
–
We are trying to keep the number to LLMs less than 20 but it’s already very difficult
-
GPT-5 with Bing Search: The New Search Engine Standard
By
–
For nearly three years the advice to "never use ChatGPT for search" has been sensible and justified… we need to retire that now GPT-5 with Bing search is a spectacularly useful search engine
-
Maximizing GPT-5 Search: Tips and Best Practices
By
–
Here's my article, which also includes tips on getting the most out of GPT-5 search