A foundational result in machine learning is that a single-layer perceptron with N^2 parameters can store at least 2 bits of information per parameter (Cover, 1965; Gardner, 1988; Baldi & Venkates) specifically: an MLP can memorize 2 N^2 things with only N^2 parameters
Single-Layer Perceptron Memory Capacity Foundational Result
By
–
