Is it possible to coordinate with China on AI governance? Critics of our proposed international agreement say no. But statements from Chinese government officials and academic figures paint a more optimistic picture:
This ML Prof told me that the hallucination rate for frontier reasoning LLMs is “next to nil” And then gave me data, only after I pushed him, showing a best-case rate of 4.6% (which of course is benchmark specific). 4.6% is not “next to nil”. Imagine if your accountant hallucinated 4.6% of the time. Or worse, your pilot. Aran Nayebi (@aran_nayebi) Have you had a chance to try the latest reasoning models? You'll see their hallucination rate is next to nil. In fact, there’s a big difference between frontier reasoning models & the base LLMs that're freely available to the public, see e.g. here: nitter.net/aran_nayebi/status/202… — https://nitter.net/aran_nayebi/status/2041249684698648922#m
The plot gets wilder: the prof's evidence for hallucinations has been allegedly solved is a chart from OpenAI showing that all models test hallucinated at least 4.6% of the time on known (therefore somewhat gameable) benchmark. That certainly isn't "solved". Imagine if your accountant hallucinated 4.6% of the time. Or your pilot. [Translated from EN to English]
Your best data are still a 4.6% hallucination rate, and on a handpicked benchmark at that. Game over. 4.6% isn’t even close to zero. And for many applications that’s deadly. Thanks for playing, and goodbye.
With curve-fitting, you are recording a lossy approximation of the output of some generative program. With symbolic learning, you are losslessly reverse-engineering the source code of the generative program. Symbolic learning won't be the best fit for all problems, but for the ones where the latent program is reasonably simple, it will outperform by many orders of magnitude.
when do we see self-improvement in AI research vs. biology? @LiamFedus, Cofounder @periodiclabs and former lead of post-training OpenAI, on @NoPriorsPod
Very exciting breaktrough: An FDA-designated AI tool called Vox can analyze just five seconds of a patient’s voice to detect signs of worsening heart failure, using patterns linked to fluid buildup that humans cannot hear. Trained on more than 3 million voice samples and supported by five clinical trials, it points to a huge shift in healthcare: cheaper, earlier, phone-based detection for a disease affecting 64 million people worldwide and costing the U.S. over $30 billion a year. I love it.
Google had the foresight to develop TPUs back in 2012. Today, they have by far the most compute. In the long term, Google is in one of the best starting positions: a solid revenue and product base, compute, and above all: distribution. Epoch AI (@EpochAIResearch) We estimate that over 60% of global AI compute is owned by the top US hyperscalers, led by Google with the equivalent of roughly 5 million Nvidia H100 GPUs! Unlike the other hyperscalers, which rely primarily on Nvidia, Google’s fleet is dominated by its custom TPU chips. — https://nitter.net/EpochAIResearch/status/2041241217334419851#m
We rebuilt how MoE models generate tokens on Blackwell GPUs, resulting in 1.84x faster inference and more accurate outputs. These improvements directly contribute to how we train Composer, allowing us to ship improved versions of the model more often.