18 years ago tomorrow, I landed in Silicon Valley to work on Google Translate team (which created the Transformer), & trained the world's largest language models at the time. Today, I'm even more excited by what SV will invent as that LM knowledge has spread to all of SV.
@reza_zadeh
-
Y2K Bug Hysteria Compared to Deceleration Concerns
By
–
Y2K bug hysteria approximates decel hysteria
-
From Skepticism to Reality: Text as Vectors
By
–
I remember so many researchers who confidently pronounced “but words/sentences/paragraphs is not a vector!!” Here we are shoving *the entire internet* into a few vectors. How it started:
-
Standing in the Way of AI Research is Unethical
By
–
Standing in the way of AI research is unethical. My keynote to all industrial leaders of Denmark at the AI Denmark summit. https://
industriensfond.dk/nyhed/det-vill
e-vel-naermest-vaere-uetisk-at-staa-i-vejen-for-udviklingen-af-kunstig-intelligens/
… -
Ilya Sutskever presents GPT-2, foundation of ChatGPT
By
–
Ilya Sutskever @ilyasut presenting GPT-2 back in March 2019, the core of ChatGPT. pic.twitter.com/9I6IUbq3JA
— Reza Zadeh (@Reza_Zadeh) 19 novembre 2023Ilya Sutskever @ilyasut presenting GPT-2 back in March 2019, the core of ChatGPT.
-
Two Misleading Analogies: Data as Oil and AI as Nuclear
By
–
Two of the most misleading analogies of past few years: – Data is the new oil – AI tech is the new Nuclear weapons tech
-
Transformers Can Learn Hierarchical Structures Through Parse Trees
By
–
Why do you say we haven’t solved “hierarchical structure”? Transformers can learn parse trees & bracketing, each of which is full generalizable to arbitrary hierarchies. I think we totally machine learn hierarchical structures now.
-
Machine Translation Research Dominates ML Field Through LLM Rebranding
By
–
Several of my old Machine Translation friends are saying that the current state of Machine Learning research feels like everybody is working on MT now. MT is eating ML by rebranding itself as LLMs.
-
LK-99 Hype and P vs NP Proof Disappointments Repeat
By
–
The LK-99 hype and letdown reminds me of the roughly once-per-decade hype over some new P vs NP proof. Heartbreaking, every time.
-
Robots Creating Art While Humans Perform Low-Wage Manual Labor
By
–
A recent twist of fate: Humans are doing the hard manual labor on minimum wage, while the robots write poetry and paint.