We (Google) were already thinking along these lines prior to the Krizhevsky et al. work. Our work at ICML 2012 (
https://
arxiv.org/abs/1112.6209) trained a neural network that was 60X larger than prior neural nets, and improved ImageNet state-of-the-art by 70% relative error, using the
Google’s 60X Larger Neural Network Achieved 70% ImageNet Error Reduction
By
–