MMCR: a new learning criterion for Self-Supervised training of joint embedding architectures based on concepts and results from statistical physics. By a team from Stanford, NYU, FAIR, and MIT.
@ylecun
-
Deep Learning Neural Networks Dismiss Critics Future Solution
By
–
Deep learning will clearly be part of the solution. To the dismay of some critics who've built their career around a futile, rear-guard, losing attempt to take down neural nets.
-
Equivariance to translations beyond linear convolution operators
By
–
Equivariance to translations implies the use of convolutions *if* we insist on using linear operators.
There are lots of non-linear operators that are equivariant. -
Exploring the Vast Space of Deep Learning Architectures
By
–
We've barely scratched the surface of the space of deep learning architectures. It's a high dimensional space, so the volume is almost entirely contained in the surface.
But we've scratched a tiny subset of the surface. -
Brain iterative processes impossible for autoregressive LLMs
By
–
Humans can spend a large amount of time thinking about a problem without outputting any "token".
In other words, there are iterative processes in the brain (e.g. implemented by recurrent networks) that are impossible to produce by an AR-LLM. -
Turing Completeness and Computational Theory: Computer Science Foundations
By
–
Yes, it's called the theory of computation, Turing completeness, and complexity theory.
In other words, the very foundation of computer science. -
Current AR-LLM Systems Lack Turing Completeness
By
–
Another way to say that current AR-LLM are not Turing complete.
-
Reasoning Planning Auto-Regressive LLMs World Models
By
–
Popping this up: a response to a question about what I consider reasoning & planning, why current Auto-Regressive LLMs can't do it, why that would require AI systems with world models, and why we still have a lot of progress to do towards AI systems that can learn and reason.
-
LLMs Limited Computation Per Token System One
By
–
Yes, I do.
LLMs produce their answers with a fixed amount of computation per token.
There is no way for them to devote more (potentially unlimited) time and effort to solving difficult problems. This is very much akin to the human fast and subconscious "System 1" decision -
Instrumental Objectives and Low-Level Guardrails in AI Design
By
–
Right, the old "instrumental objective" story.
You can have low-level guardrails against bad effects of instrumental sub-goals.
The question here is not "can you come up with a way that this could go wrong?", but rather "is there a way to do it right?" It's like turbojet design.