"The hot mess theory of AI misalignment"
a favorite talk from a recent alignment workshop turned article; offers a unique and imo fairly realistic framework for superintelligent system futures that departs from your stock paperclip maximizers.
@karpathy
-
The Hot Mess Theory of AI Misalignment Framework
By
–
-
Shoggoth Meme Critique: AI as Mirror of Human Text
By
–
imo shoggoth meme is not exactly right, I'd like to request alternate meme art. Weird choice as the "monster" is a mirror to humanity, a compression of all of our text. There are many tentacles (facets), of a diverse set of emoji. We're trying to… isolate (?) the good ones. https://
x.com/mealreplacer/s
/mealreplacer/status/1632126858299863040
… -
LLMs as Statistical Simulators of Internet Documents
By
–
A pretrained LLM is not an AI but a simulator, described by a statistical physics based on internet webpages. The system evolves given any initial conditions (prompt). To gather logprob it internally maintains a probability distribution over what kind of document it is completing
-
AI Alignment: Preventing Self-Fulfilling Prophecies of Evil Behavior
By
–
The difficulty of alignment is to a large extent the elimination of probability to role play a good AI turned evil, in spite of the vast quantities of related content we have collectively created. In this sense an unaligned AI would be a self-fullfilling prophecy.
-
AI Alignment as Entropy Control in Language Model Simulators
By
–
In particular, "good, aligned, conversational AI" is just one of many possible different rollouts. Finetuning / alignment tries to "collapse" and control the entropy to that region of the simulator. Jailbreak prompts try to knock the state into other logprob ravines.
-
LLM Psychology Framework Discussion and Analysis
By
–
More good read/discussion on psychology of LLMs. I don't follow in full but imo it is barking up the right tree w.r.t. a framework for analysis. https://
lesswrong.com/posts/D7PumeYT
DPfBTp3i7/the-waluigi-effect-mega-post
… -
Church-Turing Thesis Misunderstanding Among Critics
By
–
Agree with this; It's from people who haven't exactly internalized the Church-Turing thesis.
-
Python and English Code: Mixing Scripts with GPT API Prompts
By
–
A file I wrote today is 80% Python and 20% English. I don't mean comments – the script intersperses python code with "prompt code" calls to GPT API. Still haven't quite gotten over how funny that looks.
-
LLM Model Release: Caution on Bleeding Edge AI Research
By
–
I still am fwiw, I'm just not sure that this should apply to *bleeding edge* LLM models – they still feel new, are an active area of research, have non-trivial 2nd and 3rd order effects. I think it makes sense for some/smaller models, with some lag (e.g. GPT-2 release).
-
LLM Bias: Recognition, Challenges, and Iterative Improvement Process
By
–
the bias is widely recognized as a concerning issue and being worked on. this is not a trivial process – LLM behavior is emergent from large training datasets; tuning it involves setting up evaluations, then running experiments to improve them over time.