Any piece of content can and will be instantiated into a Q&A assistant https://t.co/4vQLvB19XO
— Andrej Karpathy (@karpathy) 20 mars 2023
Any piece of content can and will be instantiated into a Q&A assistant
@karpathy
-
Any Content Can Become a Q&A Assistant
By
–
-
VS Code Integration Benefits ChatGPT for Larger Code Tasks
By
–
Yes the integration right into VS Code removes a lot of friction… Due to this UIUX difference ChatGPT (which is otherwise more capable, esp at GPT-4) is currently better suited for larger code chunks. Would love to see this improved.
-
Copilot’s Powerful Combo Effect When Prompting Works Well
By
–
When you prompt it well enough and copilot "gets" what you're trying to achieve, it is a discrete transition that feels like doing powerful combos and dealing critical damage in video games
-
Adapting to AI: From Skepticism to Best Coding Buddy
By
–
It's really, really good. I find that many programmers still 1) haven't tried, or 2) quit too fast. It takes some time to adapt your programming habits to it and to develop internal models around when/how it is likely to work. Then it quickly becomes the best coding buddy.
-
Fine-tuning Miscalibration: Model Confidence Doesn’t Match Accuracy
By
–
If not careful, fine-tuning collapses entropy relatively arbitrarily, creates miscalibrations, e.g. see Figure 8 from GPT-4 report on MMLU. i.e., if a model gives probability 50% to a class, it is not correct 50% of the time; its confidence isn't calibrated.
-
Gradient-based Learning Becoming Less Common in AI
By
–
I'm still intuitively adjusting to the new world where gradient-based learning is less common/desirable. But the trend increases my confidence in an earlier prediction in my earlier "33 years from now" blog post https://
karpathy.github.io/2022/03/14/lec
un1989/
… -
Base LLMs as Strong Few-Shot Classifiers Without Fine-tuning
By
–
Base LLMs (non-finetuned) make very strong few-shot classifiers. Describe task in English, give few examples, read off the label probabilities on test example. No gradient-based optimization necessary. It brings a cannon to a knife fight but is fast, convenient, strong baseline.
-
MIT Licensed Evaluations and Benchmark Releases on GitHub
By
–
I didn’t work on this project personally but I feel like “undermining” is a strong word. Did you feel the same way for eg BIG-bench / HELM releases? Do you think it is good that there are more MIT licensed evals on GitHub?
-
OpenAI Open Sources Evals Framework for GPT-4 Model Evaluation
By
–
Less publicized but highly awesome aspect of GPT-4 launch was that OpenAI open sourced an evals framework, allowing us to crowdsource model evaluations at scale . The repo is getting some very high quality PRs (rewarded with GPT-4 access). I <3 evals; `pip install evals` https://
x.com/kondrich2/stat
/kondrich2/status/1635687409668227072
… -
GPT-4 livestream showcases unprecedented new capabilities for developers
By
–
The GPT-4 developer livestream (
https://
youtube.com/watch?v=outcGt
bnMuQ
…) was a great preview of new capability. Not sure I can think of a time where there was this much unexplored territory with this much new capability in the hands of this many users/developers.