One thing in AI research that I have finally recognized with clarity is the idea of “inertia bias”: continuing to do something when it’s not the best option. The most basic instance of inertia bias is the feeling of “I already spent time implementing X, so let me continue trying
@_jasonwei
-
The Adrenaline of Launching Massive GPU Training
By
–
There’s no adrenaline rush like launching a massive gpu training
-
Why Hiring More AI Researchers Can Be Counter-Productive
By
–
For most companies, hiring more people is strictly better. However, this is often not true in AI research. AI research is often bottlenecked by compute, and when this is the case, hiring more researchers can be counter-productive. I remember back at Google Brain, my manager once
-
The Trade-off Between Browsing Capability and LLM Creative Magic
By
–
Browsing is great for information retrieval and massively reduces hallucinations, but I feel that it is easy for models that browse to lose some of the “magic” of large language models. What I mean by magic is when language models give rich, organic responses reflecting the
-
Congratulations on impressive work with alignment implications
By
–
Super cool work, congrats!! Has implications on alignment
-
Why AI Researchers Should Write Unit Tests for Code
By
–
As an AI researcher there are many good reasons to write unit tests for your code, but perhaps the best motivator to write tests is the respect (and often surprise) from other people when they look at your code and see you actually wrote unit tests.
-
Leaders at OpenAI Inspire by Writing Code Alongside Their Teams
By
–
It’s inspiring to see co-founders & team leads at OpenAI (Greg being a prime example) writing code. The feeling invoked in me is almost like Medieval soldiers being inspired by the King fighting alongside them in battle (e.g., Richard the Lionheart, King Henry V, Charlemagne).
-
Hallucinations in AI: From Hidden Problem to Research Focus
By
–
It’s pretty cool how hallucinations emerged as a mainstream research topic due to ChatGPT’s ability to say “Sorry, I can’t answer that.” In the past (i.e., a few years ago), AI would always guess, so we would simply compute: accuracy = num_correct / num_examples. Now we are
-
Mixture of Objectives for Training Biological Neural Networks
By
–
Congrats Yi! Will you use mixture of objectives to train biological neural nets too?
-
Scale’s Surprising Emergent Properties in Models and Humans
By
–
“14. Scale often has surprising emergent properties.” Amen, both in language models and humans