We applied GPT-4 to interpretability — automatically proposing explanations for GPT-2's 300k neurons — and found neurons responding to concepts like similes, “things done correctly,” or expressions of certainty. We aim to use Al to help us understand Al: https://
openai.com/research/langu
age-models-can-explain-neurons-in-language-models
…
LLMS
-
GPT-4 Explains GPT-2 Neurons: Interpretability Breakthrough
By
–
-
RedPajamas and INCITE Models: New Open Source LLM Generation
By
–
And of course LLaMA isn't the only one. RedPajamas, from @togethercompute
, is based on the LLaMA training set and will likely breed a whole new generation of LLaMA-like OSS models—as well as its own RedPajama-INCITE 3B/7B models (keep an eye on that 3B one!!) -
Maximizing Power in Smaller AI Models for Local Deployment
By
–
The goal here is to squeeze as much power out of as small a model as possible. The smaller the model, the easier it is to update and tune—as well as run it cheaply (or even locally on a device).
-
Meta LLaMA turbocharges open source developer community ecosystem
By
–
Meta's LLaMA model has turbo-charged the open source developer community, even if it's still restricted to research use. I'm hearing that a full OSS version will be dropping soon and that Meta is exploring models with fewer parameters than its smallest 7B model.
-
BingChat evolves with features challenging ChatGPT and Google
By
–
#BINGChat evolves and… WOW! Here are the new features that will hurt #ChatGPT and #Google → https://youtu.be/sP-21AqAHKw
-
Prompt Engineering: Crafting Expressive Prompts for Complex Tasks
By
–
Prompt engineering boils down to writing “expressive” prompts. Explains why techniques like chain of thoughts works well for complex tasks.
-
No standardized approach yet, check Chip Huyen’s LLM engineering blog
By
–
I think there is no standardized way yet. You might find Chip Huyen blog helpful regarding that. https://
huyenchip.com/2023/04/11/llm
-engineering.html
… -
Constitutional AI: Claude’s values and constitution explained
By
–
In this post, we explain what Constitutional AI is, what the values in Claude’s constitution are, and how we chose them:
-
Constitutional AI: Encoding Values Into Language Models
By
–
How does a language model decide which questions it will engage with and which it deems inappropriate? We use Constitutional AI to more directly encode values into our language models.
-
Constitutional AI Enables Explicit Values for Language Models
By
–
Our research on Constitutional AI allows us to give language models explicit values determined by a constitution, rather than values determined implicitly via large-scale human feedback.