A good measure of progress in AI is the intelligence Sharpe ratio, a system's average intelligence across tasks divided by its standard deviation. By this measure, current AIs are still very weak because their variance is so high.
@pmddomingos
-
Reinforcement learning and dopamine research debate
By
–
Reinforcement learning was supposed to explain how dopamine works. Now even that is in doubt:
-
LLMs maximize likelihood via cross-entropy training
By
–
Yes. For one, “Bayesian inference” is not Bayesian. Every (frequentist) n-gram model uses Bayes’ theorem. For another, LLMs have high capacity and are trained to minimize cross-entropy, which is equivalent to maximizing likelihood, so it’s not surprising they produce accurate
-
Inductive inference and guarantees in AI theory
By
–
Inductive inference does evidence => theory, and these days we even have guarantees for it in many cases.
-
Attention Head as a Differentiable Multiplexer
By
–
An attention head is a differentiable multiplexer.
-
Failed AI pioneers’ paradoxical success
By
–
Geoff Hinton set out to figure out how the brain works and failed.
Andrew Ng set out to build a complete robot and failed.
Demis Hassabis set out to achieve AGI using deep RL and failed.
Yet they all succeeded. -
A critique of Good Old-Fashioned AI limitations
By
–
Good old-fashioned AI was doubly doomed: it was both anti-numbers and anti-learning.
-
Critical analysis of LLM reasoning mechanisms
By
–
Reasoning is the opposite of free association. LLMs try to do the former with the latter. It can't work.
-
OpenAI Shifts Strategic Focus Toward Enterprise AI Products
By
–
OpenAI is flailing, refocusing on enterprise products only 3 months after declaring a code red to focus on ChatGPT.