We’ve never had a satisfying answer to any question in deep learning.
MACHINE LEARNING
-
Computers as Syntactic Machines and LLMs’ Semantic Progress
By
–
Computers are syntactic machines. Large language models are a small step toward semantics.
-
GPT-3 understands ASCII art but not rasterized images
By
–
It definitely can understand small, cartoon-style ASCII art (e.g. it knows ">" is a fish and "(o_o)" is a face) but it isn't good at rasterized images. E.g. GPT-3 could do this back in August:
-
GPT-3 identifies ASCII art MNIST as ‘8’ 57%, ‘4’ absent in top-5
By
–
There's a thread going around claiming ChatGPT / GPT‑3 can recognize subjects (e.g. "a bird", "a person") in images rasterized as ASCII art. I'm skeptical. E.g., GPT‑3 identifies this ASCII-art MNIST digit as "8" 57% of the time and "4" doesn't appear in its top-5 choices:
-
Google Research Responds to AI Criticism with 2022 Achievements Overview
By
–
Google PR counterreaction begins, from Jeff Dean, reminding everyone of the extensive contributions from Google Research across LLMs, ViTs, Multimodal, and Generative text/audio/video: https://
ai.googleblog.com/2023/01/google
-research-2022-beyond-language.html?m=1
… -
Google Research Community Contributions in AI Series
By
–
A huge thanks to all of the @GoogleResearch community whose great work is represented in this post as well as the subsequent posts in the series.
-
Building a Midwit Meme Classifier for Twitter Bot
By
–
I also overengineered some aspects of it for fun… I wanted to apply what I have learned in the fast ai course, so I created a midwit meme image classifier that I integrated into the twitter bot
-
Memory-Augmented LLMs Achieve Computational Universality
By
–
Google Brain & Alberta U Paper Confirms the Computational Universality of Memory-Augmented Large Language Models https://
syncedreview.com/2023/01/18/goo
gle-brain-alberta-u-paper-confirms-the-computational-universality-of-memory-augmented-large-language-models/
… -
Cross-attention in Transformer architecture article recommendation
By
–
nice article on cross-attention as supplementary too https://
vaclavkosar.com/ml/cross-atten
tion-in-transformer-architecture
… -
Clarification on Query and Key in Attention Mechanisms
By
–
nice, exactly! 🙂
(except you're swapping q's and k's – q is the query, the "what am i looking for", k is the key, the "what do i have", and in encoder-decoder the key,value from come from side. admittedly confusing because in dictionaries the _key_ is the "lookup" information.)