SCOOP: OpenAI has come up with 5 levels to track its progress toward its goal of building AI that can do things that only humans can currently do. at an all-hands meeting this week, company execs told employees they believe they're nearing level 2.
RESEARCH
-
Estimating compute improvements from 2019 to 2024
By
–
This information was never released but I'd expect it was a lot more. In terms of multipliers let's say 3X from data, 2X from hardware utilization, in 2019 this was probably a V100 cluster (~100 fp16 TFLOPS), down from H100 (~1,000), so that's ~10X. Very roughly let's say ~100X
-
Arithmetic Operation for Layer Norm Standard Deviation Calculation
By
–
Do you see an arithmetic operation that could help us calculate this layernorm standard deviation? pic.twitter.com/nV4wwJRCLB
— Andrej Karpathy (@karpathy) 11 juillet 2024Do you see an arithmetic operation that could help us calculate this layernorm standard deviation?
-
Robot Embodiment Workshop Explores Voice Changes and Context
By
–
Thank you to the companies for supporting our workshop! Looking forward to hearing how voices might change with robot embodiment and context!
-
Digital Gap in Genomics and Wastewater Surveillance Systems
By
–
While progress has been made with genomics and wastewater surveillance, digital is the big miss.
My colleague @JayPandit put together a review paper on this topic @NatureBiotech https://
nature.com/articles/s4158
7-022-01350-x
… @ScrippsRTI @scrippsresearch @GiorgioQuer -
AI Tutors Should Outperform Traditional Learning Methods
By
–
Students with AI tutors like ChatGPT should outperform past students who relied solely on memory and limited teacher time. If they don't, we have a problem. Banning AI tools ignores their potential as stepping stones. We should measure performance: are AI-augmented students
-
GPT-2 Training Now Affordable: $672 on Single GPU Node
By
–
In 2019, OpenAI announced GPT-2 with this post: https://
openai.com/index/better-l
anguage-models/
… Today (~5 years later) you can train your own for ~$672, running on one 8XH100 GPU node for 24 hours. Our latest llm.c post gives the walkthrough in some detail: https://
github.com/karpathy/llm.c
/discussions/677
… Incredibly, the -
Digital Tracking Advances Disease Transmission Assessment Beyond Contact Tracing
By
–
One of the big missed opportunities in the pandemic was more use of digital tracking. A new @ScienceMagazine paper by @ChristoPhraser @LucaFerrettiEvo and colleagues shows its rich, real-time transmission assessment, value beyond contact tracing https://
science.org/doi/10.1126/sc
ience.adm8103
… -
Turing Test Critique and AI Evaluation Methods Discussion
By
–
Appreciate the shout out to Mystery AI Hype Theater 3000 in this recent piece: https://
technologyreview.com/2024/07/10/109
4475/what-is-artificial-intelligence-ai-definitive-guide/
… Given the discussion of the Turing Test, I also want to point to this ep of OOAC @alexhanna and I did on that topic: https://
ouropinionsarecorrect.com/shownotes/2024
/4/4/the-turing-test-is-bullshit-walex-hanna-and-emily-m-bender
… -
CRISPR Screens Reveal Molecular Switch Basis for Lupus
By
–
Discovery of a molecular switch basis for lupus (SLE) using #CRISPR screens to unravel the CXCL-13 T cell circuitry @Nature https://
nature.com/articles/s4158
6-024-07627-2
… @CalvinYLaw_ @Deepakarao @jaehyukchoimd @vwacle @NorthwesternMed