RESEARCH

→ View original post on X — @jeffdean

DistBelief convolutions development shared weights 2012

By

@jeffdean

–

06 May 2024 3h31

We supported shared weights from very early in DistBelief development, and we started adding support for various kinds of convolutions in March of 2012 (because convolutions are a very sensible idea for some things!), and you visited in July, I believe.

→ View original post on X — @jeffdean

DistBelief Convolutions Backpropagation Training Before ICML 2012

By

@jeffdean

–

06 May 2024 2h11

DistBelief supported convolutions and end-to-end training with convolutions via backprop before the ICML 2012 paper appeared (even though that particular paper didn't use convolutions).

→ View original post on X — @craigbrownphd

Mapping the Mind: Speech Production Understanding Advances

By

@craigbrownphd

–

06 May 2024 2h02

Mapping The Mind: Advances In Understanding Speech Production https://
forbes.com/sites/williamh
aseltine/2024/04/08/mapping-the-mind-advances-in-understanding-speech-production/?utm_source=dlvr.it&utm_medium=twitter
… #Innovation #ArtificialIntelligence #MachineLearning

Matrix Classes and Neural Networks: From BPTT to Modern Frameworks

By

–

06 May 2024 1h27

In the 90s I had to first implement matrix classes before I could even start coding neural nets – it was awful! I suffered so much with BPTT! Matlab did make things more efficient and was great until Python arrived. Leon Bottou and Yann Lecun pioneered SGD and torch style

Python and C++ in Computer Science Education Since 2005

By

–

06 May 2024 0h39

There is a record. We started using python for teaching in 2008 (
https://
cs.ubc.ca/~nando/340-200
8/index.html
…) but the team was on if from around 2005. Of course, some people still preferred matlab (eg @sirbayes who still loves it ). Our MoE work (PAQ8) around 2008 was all on C++ because of

Automatic Differentiation Frameworks Revolutionized Machine Learning Development

By

–

06 May 2024 0h08

To be honest, Geoff still uses Matlab and that is how we got dropout and distillation. I think software frameworks based on automatic differentiation (eg torch) changed the game, but whether it was lua, matlab or python seemed to matter less. Granted I prefer python and indeed my

DeepMind scientist advancing mathematics with AI technology

By

–

05 May 2024 23h49

Alex @SashaVNovikov is by far one of the brightest scientists I know, and his @GoogleDeepMind team is making impactful and fascinating contributions to advancing mathematics with AI. Highly recommend it!