Most neurons in language models are "polysemantic" – they respond to multiple unrelated things. For example, one neuron in a small language model activates strongly on academic citations, English dialogue, HTTP requests, Korean text, and others.
LLMS
-
Decomposing Neurons into Interpretable Features in Language Models
By
–
The fact that most individual neurons are uninterpretable presents a serious roadblock to a mechanistic understanding of language models. We demonstrate a method for decomposing groups of neurons into interpretable features with the potential to move past that roadblock.
-
LLM Named After Borges: Infinite Stories and Variations
By
–
“…The invention of a machine that can not only write stories but also all variations of every story is a significant step in the progress of humankind.” I hope someone names an LLM after Borges. Of course in some version of our history, they already have.
-
Future of AI: Task-Specific Fine-Tuned LLMs Over Large Models
By
–
"The future isn't a single large model like #ChatGPT. It’s many, many task-specific, #finetuned LLMs that are each good at a task." Check out the latest episode of Saas Scaled w/ @devvret_rishi & @arman123
. Episode: https://
pbase.ai/3RG3MjE Summary: https://
pbase.ai/46i3WSy -
LLMs Are Not Universal Turing Machines
By
–
Sigh, no. LLMs are not "approaching" universal Turing machines. They are specific programs run on computers which *are* universal Turing machines. https://
ft.com/content/5c4120
2b-6034-4cac-81a8-7968e9fa603a
… -
Mistral 7B Now Available on Poe Through Fireworks API
By
–
Mistral 7B is now available on Poe! Thanks to our API launch yesterday, Fireworks was able to quickly make this model available for all Poe users across the iOS, Android, web, and MacOS apps: https://t.co/6lbiDbTdH5 pic.twitter.com/b2Nlb9APgp
— Poe (@poe_platform) 5 octobre 2023Mistral 7B is now available on Poe! Thanks to our API launch yesterday, Fireworks was able to quickly make this model available for all Poe users across the iOS, Android, web, and MacOS apps:
-
Google WHOOPS Benchmark Assesses Multimodal Chatbot Vision
By
–
Today at 1:00 PM, visit the @ICCVConference Google booth to learn how research scientists at Google are using WHOOPS, a vision and language benchmark of synthetic and compositional images, to assess multimodal chatbots for visual commonsense.
-
Training AI Models Requires More Than Simple Code
By
–
Training a model of this type does not require merely writing 20 lines of code.
-
Internal capabilities in AI models beyond fine tuning explained
By
–
That oft-repeated phrase is technically accurate (if we ignore fine tuning) but doesn't imply anything about what capabilities are developed internally to allow that to happen.
-
First Fine-Tuned Universal Language Model Demonstrated 2017
By
–
Just happened across this video: towards the end of 2017 – this clip from the recording of http://
fast.ai lesson 4 was the first time the effectiveness of a fine-tuned universal language model was demonstrated. (Our IMDB dataset result shown at the top.)