Plot twist: it's solved or probably it's not solved or we're not sure . Really looking forward the vision capability rolling out publicly soon, unlocks a ton of new/exciting uses.
@karpathy
-
Vision Capability Breakthrough: Testing Confirms Strong Performance
By
–
We tried and it solves it :O. The vision capability is very strong but I still didn't believe it could be true. The waters are muddied some by a fear that my original post (or derivative work there of) is part of the training set. More on it later.
-
GPT-4 Released: Multimodal AI Model Now Available on ChatGPT Plus
By
–
🎉 GPT-4 is out!!
— Andrej Karpathy (@karpathy) 14 mars 2023
– 📈 it is incredible
– 👀 it is multimodal (can see)
– 😮 it is on trend w.r.t. scaling laws
– 🔥 it is deployed on ChatGPT Plus: https://t.co/WptpLYHSCO
– 📺 watch the developer demo livestream at 1pm: https://t.co/drEkxQMC9H https://t.co/WUYzwyxOqaGPT-4 is out!!
– it is incredible
– it is multimodal (can see) – it is on trend w.r.t. scaling laws
– it is deployed on ChatGPT Plus: http://
chat.openai.com
– watch the developer demo livestream at 1pm: https://
youtube.com/live/outcGtbnM
uQ?feature=share
… -
The Transformer as Its Own Attacker: An Ironic Meta
By
–
not bad except the meta is that the attacker is the Transformer itself
-
Dropout=1.0 Issue: PyTorch Should Raise ValueError
By
–
The model is not "turned off during training". With dropout=1.0, for dropout layers you'll get all zero at train and, apparently, identity at test. I don't think pytorch should have allowed dropout=1.0. It should be ValueError, not sure I get the reasoning there.
-
Dropout Layers Leak Training Phase Information in Transformers
By
–
Dropout layers in a Transformer leak the phase bit (train/eval) – small example. So an LLM may be able to determine if it is being trained and if backward pass follows. Clear intuitively but good to see, and interesting to think through repercussions of
-
The Hot Mess Theory of AI Misalignment Framework
By
–
"The hot mess theory of AI misalignment"
a favorite talk from a recent alignment workshop turned article; offers a unique and imo fairly realistic framework for superintelligent system futures that departs from your stock paperclip maximizers. -
Shoggoth Meme Critique: AI as Mirror of Human Text
By
–
imo shoggoth meme is not exactly right, I'd like to request alternate meme art. Weird choice as the "monster" is a mirror to humanity, a compression of all of our text. There are many tentacles (facets), of a diverse set of emoji. We're trying to… isolate (?) the good ones. https://
x.com/mealreplacer/s
/mealreplacer/status/1632126858299863040
… -
AI Alignment: Preventing Self-Fulfilling Prophecies of Evil Behavior
By
–
The difficulty of alignment is to a large extent the elimination of probability to role play a good AI turned evil, in spite of the vast quantities of related content we have collectively created. In this sense an unaligned AI would be a self-fullfilling prophecy.
-
AI Alignment as Entropy Control in Language Model Simulators
By
–
In particular, "good, aligned, conversational AI" is just one of many possible different rollouts. Finetuning / alignment tries to "collapse" and control the entropy to that region of the simulator. Jailbreak prompts try to knock the state into other logprob ravines.