It will be hard to empathize when your background is at an elite school where everyone naturally listens to what you have to say and takes an interest in what you do. Outside of that world, it's very very very hard to have anyone notice your work.
@jeremyphoward
-
Academic Gatekeeping: Credit for Outside Lab Researchers
By
–
Giving credit to people outside famous labs is just a fucking anathema to the academic community. Absolutely fucking gatekeeping bullshit.
-
Transfer Learning History Vision Models Fast.ai
By
–
And it's absolutely not true that everyone using vision models was using transfer learning when we starting http://
fast.ai. Absolutely false. -
ULMFiT LSTM Evolution Language Model Fine-tuning History
By
–
Right, and indeed ULMFiT was an LSTM. But also, that earlier work wasn't using language modeling on a general purpose corpus as a self-supervised task then fine-tuning that in two more steps for downstream tasks like today's LLMs. (It wouldn't have been feasible on that h/ware)
-
Crediting Alec for LLM pretraining oversimplifies research history
By
–
Alec is a once in a generational researcher, but saying that he invented pretraining is not only a bit of stretch, but it's also a disrespect to other people's work. Flowers ☾ (@flowersslop) Every LLM from any lab today traces back to this guy, who was the only person at OpenAI pushing for pretraining transformer language models. He built GPT-1. After that did others see the potential. He invented it, and almost none of the so called AI experts even know his name. — https://nitter.net/flowersslop/status/2037892926785634720#m
→ View original post on X — @jeremyphoward, 2026-03-29 12:21 UTC
-
Recognition Challenges for AI Researchers Outside Major Labs
By
–
If you're not at a big lab, you have to fight to get noticed. And even then…
-
LLMs Missing from Wikipedia and Academic Historical Records
By
–
And yet still no mention on Wikipedia or nearly any article or book about the history of LLMs or AI.
-
ULMFiT Language Modeling Self-Supervised Pre-training Innovation
By
–
For sure. And indeed CV pre-training was a key inspiration for ULMFiT. AFAIK there wasn't previous examples of using language modeling on a general purpose corpus as a self-supervised task then fine-tuning that in two more steps for downstream tasks (i.e like today's LLMs).
-
Alec’s Research: Scaling Existing AI Work to New Architecture
By
–
Alec is amazing, but he says himself in his paper that he was simply taking our existing work, and moving it to a new arch and scaling it.
-
ULMFiT Transfer Learning: The True Foundation of Modern LLMs
By
–
Every LLM from any lab today, including from OpenAI taces back to @jeremyphoward and @seb_ruder with their ULMFiT paper. The breakthrough for LLMs was transfer learning, not Attention. Flowers ☾ (@flowersslop) Every LLM from any lab today traces back to this guy, who was the only person at OpenAI pushing for pretraining transformer language models. He built GPT-1. After that did others see the potential. He invented it, and almost none of the so called AI experts even know his name. — https://nitter.net/flowersslop/status/2037892926785634720#m
→ View original post on X — @jeremyphoward, 2026-03-28 20:34 UTC