The reality of the situation is that training cutting edge AI demands tens of thousands of GPUs and their active, expert maintenance. Every company that wishes to be at the bleeding edge of AI will either need to build this (insane hard), or partner to some extent.
@karpathy
-
Open Source AI: Balancing Innovation with Accessibility and Safety
By
–
lol meme; my current thinking is it's not clear that fully open source for *bleeding edge* AI is desirable (or sustainable). OpenAI is imo doing a lot by making AI (APIs, ChatGPT, CoPilot, …) available and dirt cheap. A lot of work (e.g. bias) still needed and actively ongoing.
-
GPT Implementation in 60 Lines of NumPy and JAX
By
–
Late to the party but "GPT in 60 Lines of NumPy" / picoGPT is nicely done: https://
jaykmody.com/blog/gpt-from-
scratch/
…
– good supporting links/pointers
– flexes some of the benefits of JAX: 1) trivial to port numpy -> jax.numpy, 2) get gradients, 3) batch with jax.vmap
– inferences gpt-2 checkpoints -
Helpful Links for Trending ML Projects and Papers
By
–
helpful links i am aware of for trending projects:
1. papers: https://
papers.labml.ai/papers/weekly
2. papers+code: https://
paperswithcode.com
3. code: https://
github.com/trending -
Andrej Karpathy discontinues arxiv-sanity maintenance
By
–
Sad but I just don't have the time to maintain it anymore. It's possible I'll try to build yet another version of a more LLM-powered arxiv-sanity, I have a few ideas there. For now it is what it is sorry. Please refer to:
1 https://
papers.labml.ai/papers/weekly
2 https://
paperswithcode.com -
New Programming Paradigm Expands Developer Base to 1.5B People
By
–
This is not an exhaustive list (people can add more in replies), but at least some of the articles I saw recently that stood out. It's still early days but this new programming paradigm has the potential to expand the number of programmers to ~1.5B people.
-
GPTs Execute Natural Language Programs Through Document Completion
By
–
9/ Pulling in one more relevant tweet of mine from a while ago. GPTs run natural language programs by completing the document.
-
Bing Chat’s Prompt Revealed by Injection, Identity Construction
By
–
7/ The prompt allegedly used by Bing chat, potentially spilled by a prompt injection attack https://x.com/marvinvonhagen/status/1623658144349011971?lang=en … important point for our purposes is that the identity is constructed and programmed in English, by laying out who it is, what it knows/doesn't know, and how to act.
-
GPT as Backend: LLM-Powered State Management with Natural Language
By
–
6/ "GPT is all you need for the backend" https://
github.com/TheAppleTucker
/backend-GPT
…
Tired: use an LLM to help you write a backend
Wired: LLM is the backend
Inspiring project from a recent Scale hackathon. The LLM backend takes state as JSON blob and modifies it based on… English description.