Memorization Capacity of Deep Neural Networks under Parameter Quantization in 2019 these folks measured the capacity of RNNs/CNNs/MLPs, also using this class-based entropy idea they quantized the networks to half-precision and saw a capacity drop (but not nearly by half)
MACHINE LEARNING
-
Single-Layer Perceptron Memory Capacity Foundational Result
By
–
A foundational result in machine learning is that a single-layer perceptron with N^2 parameters can store at least 2 bits of information per parameter (Cover, 1965; Gardner, 1988; Baldi & Venkates) specifically: an MLP can memorize 2 N^2 things with only N^2 parameters
-
Capacity Measurements for Language Models Research
By
–
in prep for our new research dropping on ArXiv tomorrow (i think), here is a thread about…. CAPACITY MEASUREMENTS FOR LANGUAGE MODELS
-
Meta-Reinforcement Learning Tutorial: A Comprehensive Guide
By
–
A Tutorial on Meta-Reinforcement Learning Beck et al.: https://
arxiv.org/abs/2301.08028 #ArtificialIntelligence #MetaLearning #ReinforcementLearning -
AlphaEvolve: LLMs and Evaluators Discover New Algorithms
By
–
Why AlphaEvolve matters This isn’t just about math. AlphaEvolve shows: – LLMs + evaluators can discover new knowledge
– Code generation can evolve into true algorithm innovation
– AI agents can now tackle hard science problems autonomously -
AlphaEvolve Solves 300-Year Kissing Number Problem in 11D
By
–
One highlight: Kissing number problem This 300-year-old geometry challenge asks:
“How many spheres can touch one central sphere in 11D space?” AlphaEvolve pushed the known lower bound from 564 → 593. That’s historic. -
AlphaEvolve Discovers Better Matrix Multiplication Algorithms
By
–
It designs new math AlphaEvolve discovered better algorithms for 4×4 matrix multiplication — improving on a record that stood since 1969. It also tackled over 50 open problems in math, and in 20% of them, it found better solutions than humans. Yes, really.
-
AlphaEvolve: Darwin-Inspired Code Evolution with Gemini Models
By
–
Built with Gemini: AlphaEvolve uses a model ensemble: – Gemini Flash for rapid idea generation
– Gemini Pro for deeper, high-quality code suggestions Each candidate program is evaluated, scored, and evolved Darwin meets compilers. -
AI advancement rivals entire digital revolution next decade
By
–
It fascinates me that RIGHT NOW – this exact moment – we're living through the most transformative decade in human history. The last 5 years saw AI advance more than computing did in its first 50 years. The next 10 years could rival the entire digital revolution from 1950
-
Why Language Models Build Rich Knowledge Hierarchies
By
–
…which is literally why I created LLMs. I explained in the original videos that the entire reason was because language models, to be good at their job, must build a rich hierarchy of abstractions and capabilities and implicit world-knowledge. Turns out it works! 😀