Very excited to finally share OpenAI's "deep research" model, which achieves twice the score of o3-mini on Humanity's Last Exam, and can even perform some tasks that would take PhD experts 10+ hours to do! A few thoughts on the implications: Deep research can be seen as a new
@_jasonwei
-
Early AI Adoption and Knowledge Retention Beyond the LLM Era
By
–
i got into AI in 2017 thank you very much also "all the work went to the garbage after the llm era, but all the grinding and learning still stayed within me" lol
-
The Dopamine Cycle of Daily AI Research Experiments
By
–
Somewhat meta but there is a dopamine cycle in doing AI research that is pretty interesting Every day you wake up and you think about what experiment to run. You think thing X matters so you decide to improve it or ablate it. Then you write the code and pay some compute to find
-
RL Optimization Algorithm Magic in Perfect Environment Convergence
By
–
Magic is what happens when an unstoppable RL optimization algorithm powered by sufficient compute meets an unhackable RL environment
-
DeepMind Paper on Document-Based Factuality Evaluation
By
–
Nice paper from Deepmind takes a fresh angle on factuality: http://
arxiv.org/abs/2501.03200 While most existing factuality datasets focus on public world knowledge, this paper evaluates whether responses are consistent with a provided document as context. This is an elegant and -
Chain-of-Thought Reasoning Effectiveness in Math and Symbolic Tasks
By
–
Reference above: "To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning" https://
arxiv.org/abs/2409.12183 -
Dataset Selection: An Underrated Skill in Modern AI Research
By
–
An underrated but occasionally make-or-break skill in AI research (that didn’t really exist ten years ago) is the ability to find a dataset that actually exercises a new method you are working on. Back in the day when the bottleneck in AI was learning, many methods were
-
Five Years of AI: Key Technical Lessons in Language Models
By
–
Reflecting back, these were the biggest technical lessons for me in AI in the past five years: 2020: you can cast any language task as sequence prediction and learn it via pretrain + finetune 2021: scaling to GPT-3 size enables doing arbitrary tasks specified via instructions
-
Internet Access More Effective Than Hallucination Research for Models
By
–
Realization: the old style of “hallucinations research” via self-calibration is probably going to die down. I used to be very excited about it but now I am skeptical because giving models internet access (e.g., searchGPT, perplexity) is turning out to be way higher ROI. When