It does work. With stacks of pre-trained sparse convolutional autoencoders, we can achieve a strong starting point for fine-tuning on small labeled datasets (like Caltech 101, which had 30 training samples per category) and reach near-state-of-the-art performance.
@ylecun
-
Revisiting Sparse-Regularized Autoencoders
By
–
Back when we believed deep network pre-training could be achieved using stacked autoencoders trained with group sparsity regularization.
-
Researcher Claims Priority on World Models Concept Since 2016
By
–
I started using the concept in 2016 (e.g. in my NIPS 216 keynote, in which I called it a "world simulator"). I published papers on video prediction in 2016.
This was meant to be a key step to train world models.
Ha&Schmi appeared in 2018. The slide below is from a talk I gave -
Transformers History: From Attention Mechanisms to Modern AI
By
–
Transformers were published in 2017 (by Google), but was based on work on attention and associative memory by University of Montreal and FAIR 2 years earlier. And they use neural nets and backprop that were popularized in the late 1980s. Oh, and GPUs are based on work by Bill
-
Catholic University Leadership Raises Governance Concerns
By
–
With a chair and vice-chair from the Catholic University of America? I doubt it.
-
Scientific breakthroughs precede industrial innovation by decades
By
–
Innovation from industry is enabled by scientific breakthroughs that precede it by 5, 10, or 20 years.
-
AI Self-Harm: The Prefrontal Cortex Problem
By
–
Shooting oneself in the foot?
Nope.
Shooting oneself in the prefrontal cortex. -
Tesla Model S FSD Hardware 4 Autonomous Driving Capabilities
By
–
I own a Tesla Model S with FSD and HW4. WDYM?
-
Deep Learning: ConvNets and ViTs Training with Backpropagation
By
–
Deep learning. Largely ConvNets or ViTs trained with backprop
-
AI Content Moderation Legal Requirements for Social Networks
By
–
Perhaps you would prefer one of these alternatives:
1. That these toxic contents not be filtered at all by social networks.
2. That these same people filter these contents manually and not by AI. Option 1 is illegal in many countries. Social networks are required by law to