Today GANs got 1 of 2 test of time awards at @NeurIPSConf . Many thanks to my co-author @dwf for preparing and presenting the talk. It’s no easy task to summarize 85k papers in 12 minutes
@goodfellow_ian
-
Game Theory Analysis of Intuited Algorithm in Lab Research
By
–
From there I started asking around if anyone in the lab knew game theory well enough to attempt an analysis of the algorithm I had intuited, and fortunately Jean was there
-
Game Theory Expertise Crucial to AI Problem Solving
By
–
One of the craziest pieces of luck in this whole story was that Jean was there. Jean was an intern and had a background in game theory. None of the rest of us did. I knew enough from Tim Roughgarden’s optimization class that this was a game theory problem.
-
Convex Analysis in Deep Learning: LISA Lab’s Hidden Foundation
By
–
It amused me more than once that it was important to know convex analysis in LISA lab even though deep learning people at the time tended to regard anything convex as “the other faction.” The Boyd book chapter 2 served me well more than once
-
Simplified KL Divergence Proof Advances Machine Learning Theory
By
–
Another big step at this part of the story was moving from a more complicated proof based on functional derivatives to the simpler one based on convexity of KL divergences that we published. @dwf contributed this final step
-
The Golden Era of Deep Learning: 2009-2012 Mystery
By
–
GANs was kind of late in the game for me! I had already handed in my thesis, and by that time deep learning was basically working. I really enjoyed the intellectual atmosphere of ~2009-2012 when deep learning didn't really work and all bets were open, solving it was a mystery.
-
GANs Win Test of Time Award: Reflections on 2012-2014
By
–
My GAN co-author Sherjil Ozair has written about some memories of 2012-2014 in the context of GANs winning one of this year's test of time awards, worth a read for the nostalgia if you were around back then, or for learning what it was like if you weren't
-
Contrarian approach to proving novel ML technique viability
By
–
They thought it wouldn't work, and I was contrarian enough to go home and code it up to prove that it would. But it's a good thing we didn't know about MMD (kernelized moment matching) or we would have just done that instead of inventing something new.
-
Scaling Machine Learning: Third Order Moments vs Discriminator Approach
By
–
At the party, the people on the generator / moment matching party asked me for software engineering advice: how to scale up to third order moments? I said this was doomed to fail and counterproposed the discriminator idea.
-
Deep Learning Textbook Work During Thesis Transition Period
By
–
I would add here, I was not part of the generator / moment matching project, I was in lame duck mode having handed in my thesis and was mostly writing the Deep Learning textbook from home, not doing research.