In particular, "good, aligned, conversational AI" is just one of many possible different rollouts. Finetuning / alignment tries to "collapse" and control the entropy to that region of the simulator. Jailbreak prompts try to knock the state into other logprob ravines.
@karpathy
-
LLM Psychology Framework Discussion and Analysis
By
–
More good read/discussion on psychology of LLMs. I don't follow in full but imo it is barking up the right tree w.r.t. a framework for analysis. https://
lesswrong.com/posts/D7PumeYT
DPfBTp3i7/the-waluigi-effect-mega-post
… -
Church-Turing Thesis Misunderstanding Among Critics
By
–
Agree with this; It's from people who haven't exactly internalized the Church-Turing thesis.
-
Python and English Code: Mixing Scripts with GPT API Prompts
By
–
A file I wrote today is 80% Python and 20% English. I don't mean comments – the script intersperses python code with "prompt code" calls to GPT API. Still haven't quite gotten over how funny that looks.
-
LLM Model Release: Caution on Bleeding Edge AI Research
By
–
I still am fwiw, I'm just not sure that this should apply to *bleeding edge* LLM models – they still feel new, are an active area of research, have non-trivial 2nd and 3rd order effects. I think it makes sense for some/smaller models, with some lag (e.g. GPT-2 release).
-
LLM Bias: Recognition, Challenges, and Iterative Improvement Process
By
–
the bias is widely recognized as a concerning issue and being worked on. this is not a trivial process – LLM behavior is emergent from large training datasets; tuning it involves setting up evaluations, then running experiments to improve them over time.
-
Training Cutting Edge AI Requires Massive GPU Infrastructure and Expertise
By
–
The reality of the situation is that training cutting edge AI demands tens of thousands of GPUs and their active, expert maintenance. Every company that wishes to be at the bleeding edge of AI will either need to build this (insane hard), or partner to some extent.
-
Open Source AI: Balancing Innovation with Accessibility and Safety
By
–
lol meme; my current thinking is it's not clear that fully open source for *bleeding edge* AI is desirable (or sustainable). OpenAI is imo doing a lot by making AI (APIs, ChatGPT, CoPilot, …) available and dirt cheap. A lot of work (e.g. bias) still needed and actively ongoing.
-
GPT Implementation in 60 Lines of NumPy and JAX
By
–
Late to the party but "GPT in 60 Lines of NumPy" / picoGPT is nicely done: https://
jaykmody.com/blog/gpt-from-
scratch/
…
– good supporting links/pointers
– flexes some of the benefits of JAX: 1) trivial to port numpy -> jax.numpy, 2) get gradients, 3) batch with jax.vmap
– inferences gpt-2 checkpoints