Convolution is equivariant to translations.
Self-attention is equivariant to permutations.
They both have a role to play.
Conv is efficient for signals with strong local correlations and motifs that can appear anywhere.
SelfAtt is good for "object-based" representations where
INNOVATION
-
Convolution Equivariance vs Self-Attention Permutation Properties
By
–
-
Genius Makers Book Explores Ilya Sutskever OpenAI History
By
–
And definitely read Genius Makers by @CadeMetz if you want a deeper history on Ilya and OpenAI.
-
Ilya Sutskever: OpenAI Cofounder’s Impact on Generative AI
By
–
Ilya Sutskever (
@ilyasut
) is certainly one of the lesser known @OpenAI cofounders outside of tech circles, but his contributions to the advancements of AI over the last decade are significant. He has played a leading role in the development and spread of generative AI. And now -
Generations in Internet Software Conferences: Ecosystems Overlap
By
–
Free essay prompt for anyone who wants it: talk about the generations involved in Internet-based software conference, the overlaps, the mutual illegibility, the provider ecosystems, etc.
-
The Original Indie Hacker Communities Before IndieHackers Era
By
–
There was a thriving community on Joel Spolsky’s Business of Software forum who were all grinding things out in obscurity-at-large but publicly to each other, incidentally. It was IndieHackers before IndieHackers gave that general activity a name. (And before that, shareware.)
-

Columbia Seligman’s Paul Wick on Tech Investment Picks
By
–
Please enjoy my interview with Columbia Seligman's Paul Wick The tech hunter: Seligman’s Paul Wick offers picks and pans One of the most seasoned fund managers in technology, Paul Wick of Columbia Seligman has a sharp eye for nonsense in tech, but also a deep grasp of the
-
Great companies optimize for speed and continuous learning
By
–
Great companies / organizations optimize for moving quickly and learning. If you hear too often that we can’t try that for XYZ reason, you are likely not in the right place.
-
Vote for Your Favorite GEN:48 AI Films Now
By
–
Vote now for your favorite three GEN:48 films at https://t.co/BqT4MubFNb
— Runway (@runwayml) 27 octobre 2023
Voting ends at 11:59 pm EST on 10/30 pic.twitter.com/yuF5USXvAoVote now for your favorite three GEN:48 films at http://
gen48.runwayml.com Voting ends at 11:59 pm EST on 10/30 -

AI Revolutionizes Scientific Research Across 99% of Fields
By
–
AI has come a long way since the '60s. Fast forward to 2023, and over 99% of research areas are tapping into AI, thanks to its rapid proliferation. Now, scientists have a powerful research assistant at their fingertips, making complex tasks a breeze. Source: @CSIRO
-

GPT-4 Practical Applications and Real-World Use Cases
By
–
Good morning fellow AI enthusiast! This week's iteration focuses on something I didn't discuss enough in my videos: what can actually be done using GPT-4!