A very interesting text from OpenAI about hallucinations: AI models shine on leaderboards—but often not in practice. The reason: benchmarks usually only measure accuracy. Anything that is not correct—whether it is a clear error or an honest “I don't know”—is scored with 0
RESEARCH
-
User satisfaction matters more than AI evaluation benchmarks
By
–
Users don't give a shit how well you do on evals. They care how well it solves their problems. You can often get evals to *somewhat* correlate with user problems, but coverage is usually mid at best and it's a moving target. We've seen far better success w/ A/B tests against $.
-
Will AI Make Revolutionary Discoveries by 2030?
By
–
will still be making discoveries like this in 2030? https://t.co/BZBXlxLAff
— Gary Marcus (@GaryMarcus) 6 septembre 2025will still be making discoveries like this in 2030?
-
Evals Matter for Frontier Models, Not Product Builders
By
–
Let’s settle the evals debate. If you’re a lab, building general-purpose frontier models, yes, evals matter. If you’re a product builder, evals mostly don’t matter. It’s that simple.
-
AI Supercharges Data Science Workflow for Scientists
By
–
How #AI is supercharging #DataScience flow for #DataScientists :
-

IoT as Internet of Context: Essential Reading for Digital Transformation
By
–
#IoT is the #InternetOfContext
…
(1) The Age of #Context: http://
amzn.to/34utTz7
+
(2) Outside Insight: http://
amzn.to/317XV9L
——————
#IIoT #Industry40 #BigData #DataScience #AI #MachineLearning #DigitalTransformation #AlternativeData #DecisionScience -
AGI Timeline Challenges: Core Cognitive Problems Remain
By
–
8-15 and we might reach an agreement. 4 still seems awfully unlikely to me. to many core cognitive problems aren’t really being addressed, and solutions may take a while to roll once we find the basic insights we are lacking. but it’s a fair question.
-
A Promise That Brings AI to Life Responsibly
By
–
A Promise That Brings AI to Life Responsibly
#AI #AIio #AIInnovation #ML #DataScience #Futureofwork @HaroldSinnott @fogoros @iainljbrown @NandoDF @katecrawford @drhassanrashidi @YuHelenYu -
Curriculum Learning Makes Comeback Without Random Sampling
By
–
curriculum learning will make a comeback. unfortunately, random sampling is not the way heard it here first
-
Learning Style Representations in Continuous High-Dimensional Spaces
By
–
they are randomly sampled from a continuous high dimensional space that's been trained to learn what elements of imagery constitite the concept of style
