The three laws of machine learning:
1. No free lunch
2. More is different
3. Correlation is not causation
@pmddomingos
-
Three Fundamental Laws of Machine Learning
By
–
-
LLMs Make Unique Mistakes Unlike Human Reasoning
By
–
Not really. LLMs screw up on some questions like no human ever would, e.g.: https://
economist.com/by-invitation/
2022/06/09/artificial-neural-networks-today-are-not-conscious-according-to-douglas-hofstadter
… -
Intelligence lies in optimization methods, not goals
By
–
Intelligence is not in the objective function, it’s in how you optimize it.
-
Building and Refining Your World Model Over Time
By
–
Yes, you definitely should have a world model, but you learn/refine it as you go along.
-
Actor-Critic Reinforcement Learning as Backpropagation Through Time
By
–
So actor-critic RL is backprop through time.
-
Improving Sequential Decision Making by Incorporating Backpropagation Principles
By
–
But this is well below that. Let me put it another way: how can we improve RL by making it more like backprop? (And by RL I mean sequential decision making, not the current set of techniques for doing it.)
-
Reinforcement Learning and Backpropagation: A Dynamic Programming Perspective
By
–
You’re taking it too literally. I’m trying to point to a way to improve RL by recognizing that at a certain level of abstraction it’s doing something similar to backprop (namely, efficiently propagating later results back to earlier choices via dynamic programming).
-
Difficulty Distinguishing Misinformation Studies from Misinformation
By
–
It’s getting harder and harder to tell misinformation studies from misinformation.
-
Difficulty Distinguishing Misinformation Studies From Misinformation
By
–
It’s getting harder and harder to tell misinformation studies from misinformation.
-
Gradient Descent and Graduate Student Descent in Deep Learning
By
–
The inner loop of deep learning is gradient descent. The outer loop is graduate student descent.