9/ If we can solve the supply chain issues, the future for our models is bright. We could expect continued progress with each successive generation for many years to come.
LLMS
-
Data Scarcity and Building Data Abundance for AI
By
–
7/ On data, it's a bit more complicated. We've exhausted most of the internet as pre-training data, and as previously mentioned, the gains are coming from post-training. The next 4 years will need us to overcome data scarcity and build data abundance.
-
GPT-3 Breakthrough: Scaling Language Models Revealed AI Potential
By
–
2/ GPT-3 was when it first became clear what the potential of scaling language models was. The efficacy of GPT-3 took the AI community by surprise for the most part—the capabilities were staggering compared to everything that came before in NLP.
-
GPT-3 Marked Inflection Point for Language Models AI
By
–
3/ @scale_AI had started working on language models the year before on the very first RLHF experiments on GPT-2. But GPT-2 still felt very much like a toy at that time. GPT-3 was the first moment where it was obvious this would be the major theme of AI
-
OpenAI’s 2020 Scaling Laws Paper: Foundation of Modern AI
By
–
4/ The key insight of scaling laws was actually published earlier that year in Jan 2020 by OpenAI—"Scaling Laws for Neural Language Models" This paper and GPT-3 ultimately was the start of the current AI era of rapidly scaling language models—especially the compute + data needed
-
GPT-3 Anniversary: Four Years of Language Model Evolution
By
–
1/ Today is the 4th anniversary of the original GPT-3 paper—"Language Models are Few-Shot Learners" Some reflections on how the last 4 years have played out, and thoughts about the next 4 years
-
Anthropic testing refreshed UI and sidebar layout for chat interface
By
–
Anthropic seems to be preparing a refreshed UI for its "new chat" page This change is part of a layout redesign where chat history moves to the side panel. Does it look better to you?
-
Academic AI researchers losing independence studying industrial LLMs
By
–
Academic AI researchers are slowly committing suicide by reducing what they do to studying and tweaking industrially produced LLMs.
-
Claude AI Implements EU Election Information Warning Banner
By
–
Claude AI is rolling out a new EU Parlament elections warning This banner indicates that Claude AI cannot provide accurate information on this topic and refers to the official resource.
-
HuggingFace Integrates Command R+ Tools in HuggingChat
By
–
Congrats to @HuggingFace for making tools available in HuggingChat! We're happy that Command R+ was chosen to power the experience.