most foundational concept in deep learning that no one understands is probably the Neural Tangent Kernel (NTK) this line of work studies neural networks of *infinite width*, which explain a lot about normal finite-width NNs and there is exactly one Very Good blog post on them:

→ View original post on X — @jxmnop,