Congrats, Kenneth! It was awesome teaming up with you to craft great developer experiences at Stripe. Can’t wait to see where your explorations at the intersection of LLMs and developer tools take you!
INNOVATION
-
MacArthur Fellow Recognition for AI Researcher lrhutyra
By
–
Congratulations to @lrhutyra for being a 2023 MacArthur Fellow! Learn more ↓ #MacFellow https://
x.com/macfound/statu
/macfound/status/1709605154759065684
… -
Stripe API design influenced OpenAI architecture decisions
By
–
Because @gdb built the stripe API and then started OpenAI so went with the design that worked well there.
-
Sparse Coding and Distributed Representations in Machine Learning Research
By
–
Our research builds on work on sparse coding and distributed representations in neuroscience, disentanglement and dictionary learning in machine learning, and compressed sensing in mathematics.
-
Monosemantic Features and Sparse Autoencoders for Neural Networks
By
–
There's a lot more in the paper if you're interested, including universality, "feature splitting", more evidence for the superposition hypothesis, and tips for training a sparse autoencoder to better understand your own network! https://
transformer-circuits.pub/2023/monoseman
tic-features/index.html
… -

Finite-State Automata Features Generate Complex HTML Behaviors
By
–
Features connect in "finite-state automata"-like systems that implement complex behaviors. For example, we find features that work together to generate valid HTML. https://
transformer-circuits.pub/2023/monoseman
tic-features/index.html
… -

Sparse Autoencoders Extract Purer Concepts Than Individual Neurons
By
–
Dictionary learning works! Using a "sparse autoencoder", we can extract features that represent purer concepts than neurons do. For example, turning ~500 neurons into ~4000 features uncovers things like DNA sequences, HTTP requests, and legal text. https://
transformer-circuits.pub/2023/monoseman
tic-features/index.html
… -
LLM Named After Borges: Infinite Stories and Variations
By
–
“…The invention of a machine that can not only write stories but also all variations of every story is a significant step in the progress of humankind.” I hope someone names an LLM after Borges. Of course in some version of our history, they already have.
-
Life as a System: Exploring Biological Complexity
By
–
The study of life as a system is so fascinating.
-
Future of AI: Task-Specific Fine-Tuned LLMs Over Large Models
By
–
"The future isn't a single large model like #ChatGPT. It’s many, many task-specific, #finetuned LLMs that are each good at a task." Check out the latest episode of Saas Scaled w/ @devvret_rishi & @arman123
. Episode: https://
pbase.ai/3RG3MjE Summary: https://
pbase.ai/46i3WSy