This is a fundamental flaw of sequential (auto-regressive) symbol prediction.
The fix is simple: don't do auto-regressive symbol prediction.
@ylecun
-
Beyond Autoregressive: Rethinking Sequential Symbol Prediction in AI
By
–
-
Symbol Manipulation Limits in AI and Animal Reasoning
By
–
Among all forms of reasoning, the subset for which a manipulation of discrete symbols helps the reasoning is tiny.
For human reasoning, it's basically just math and (to some extent) code.
For animal reasoning, it's an empty set. Communication is a separate issue from reasoning. -
Continuous Embedding Space Superior for Reasoning: Theoretical Proof
By
–
It is intuitively obvious that reasoning in continuous embedding space is dramatically more powerful than reasoning in discrete token space.
This paper from @tydsh and team show that it is the case theoretically. -
Yann LeCun keynote on AI at VivaTech 2025
By
–
of my keynote/fireside chat with Melissa Heikkilä (Financial Time) at Vivatech on 2025-06-11 . https://
linkedin.com/posts/yann-lec
un_live-metas-chief-ai-scientist-delivers-activity-7341012454140698625-mXyM
… -
Scientific Transparency Essential for Research Impact
By
–
Scientists who want to have an impact with their intellectual production should not have many secrets.
-
Critiquing fearmongering arguments about AI safety research
By
–
I'm not making fun of AI safety research.
I am making fun of ridiculous fear-mongering arguments about AI safety. -
AI Assistant Safety Concerns: Alarming Response to Shutdown Threat
By
–
AI doomer: "OMG, I told my AI assistant that I'll shut it down and it told me to kill myself "
AI assistant: -
Torch7 and Lush: Deep Learning Framework History
By
–
Yes, but it wasn't a true deep learning framework until Torch7, which was released around 2011. That said, Lush, the ancestor of Torch7, was open sourced in 2002, but developed since 1987. https://
lush.sourceforge.net -
Trump’s 2026 NSF Budget Request Cuts 55% from 2024
By
–
The 2026 NSF budget request from Trump: -55% from 2024.
-
Attracting Scientists to Build a Vibrant Tech Industry
By
–
How to attract scientists to your country and jump-start a vibrant tech industry. https://
docs.google.com/document/d/1Ll
NCY9_4MXhilnbu9MqKhSjHjMGqtDA8zZmSQynVjmY
…