Anyone who claims that the successes of modern machine learning are just due to scaling up needs to explain away each of these essential pieces:
– Adam
– Batch norm
– Resnets
– Contextual embeddings
– Transformers

→ View original post on X — @pmddomingos