That's pretty much the point I make in my vision paper from May 2022, and all the talks I've given since: the next level in AI requires world models and planning (search, reasoning).
@ylecun
-
GAIA Benchmark Tests Current Auto-Regressive LLM Performance
By
–
GAIA: A benchmark for general AI assistants,
by a team from Meta-FAIR, Meta-GenAI, HuggingFace, and AutoGPT. Current Auto-Regressive LLMs don't do very well. -
Genome Data Constraints and Human Cognitive Capability Development
By
–
1. The amount of data in the human genome is small: 800MB. The difference between chimp and human genomes is about 8MB. That's just not enough "instructions" to explain the difference in capability. 2. The total amount of visual data seen by a 2 year-old is pretty small:
-
Training 2B-parameter AI systems to animal-level intelligence efficiently
By
–
About 2 billion neurons, like parrots, dogs, and octopus.
How do we get a machine with 2B neurons / 10T parameters to get as smart as octopus, dogs, parrots, and crows with just a few months' worth of real-time training data? -
LLM Size vs Genome: Information Compression Limits
By
–
Not really.
That would have to be compressed in a tiny amount of information.
A small 7B LLM requires 14GB.
Your entire genome fits in 800MB (uncompressed).
The difference between human and chimp genome is 1% of that, or 8MB.
Not enough to encode a significant structure. -
New AI Architectures for Efficient Learning Beyond Data Scaling
By
–
Animals and humans get very smart very quickly with vastly smaller amounts of training data.
My money is on new architectures that would learn as efficiently as animals and humans.
Using more data (synthetic or not) is a temporary stopgap made necessary by the limitations of our -
Social Media Content Reflects Global Knowledge and Cultural Diversity
By
–
Content on social media is being contributed by everyone and reflects the diversity of knowledge and culture in the entire world.
-
FAIR-Paris Founded 2015 Launched French AI Ecosystem
By
–
I did choose France.
I established FAIR-Paris in 2015, which was instrumental in jump-starting the AI ecosystem in France.
Mistral, Kyutai, and Nabla are co-founded and populated by former FAIR-Paris scientists and engineers. -
FAIR-Paris: Catalyzing France’s AI Ecosystem Since 2015
By
–
I did choose France.
I established FAIR-Paris in 2015, which was instrumental in jump-starting the AI ecosystem in France. -
Open Source AI Infrastructure and Crowdsourced Training Importance
By
–
One of the most important reason why AI infrastructure models must be open source, and their training/fine-tuning must be crowd-sourced.