RESEARCH

Commercial airlines crash at a rate of 7 per 41 million flights. Current top LLMs hallucinate (by one estimate) 4.6% of the time, about once in 25 prompts. On a known benchmark (usually things are worse on new benchmarks). Comparing them, like the guy below does, is ludicrous. If commercial airlines crashed at the rate that LLMs hallucinated it would be 1.87 million crashes per 41 million flights. Around a quarter million times greater. Carlo (@bluberino123) “current models still hallucinate” is true in the same way “planes still crash” is true. broad enough to be technically correct, too broad to settle the actual argument. the real question is whether you’re following the frontier closely enough to say where the failure lives now. and honestly, that’s why aran’s criticism lands for me. i think you’re sincere in the belief, because i’ve read things from you outside x and i don’t think the concern is fake. but judged only from your x feed, you often come off bitter, like this has become personal. you keep citing papers using outdated models, posting gotcha screenshots, and using non-reasoning models as stand-ins for the frontier, and after a while it starts to look like you’re painting a lower-resolution picture of the field so your earlier predictions still look right. — https://nitter.net/bluberino123/status/2041307995116564935#m

→ View original post on X — @garymarcus, 2026-04-07 02:21 UTC

Another way to think about, with some basic math: Gary Marcus (@GaryMarcus) This is utterly false — by many order of magnitudes. Current top LLMs hallucinate (by one estimate) 4.6% of the time, about once in 25 prompts. On a known benchmark (usually things are worse on new benchmarks). Commercial airlines crash at a rate of 7 per 41 million flights. If commercial airlines crashed at the rate that LLMs hallucinated it would be 1.87 million crashes per 41 million flights. Around a quarter million times greater. The dude below does not know what he is talking about. — https://nitter.net/GaryMarcus/status/2041339541701939365#m

→ View original post on X — @garymarcus, 2026-04-07 02:19 UTC