“Making A.G.I. claims can be a reputation killer for computer scientists. What one researcher believes is a sign of intelligence can easily be explained away by another, and the debate often sounds more appropriate to a philosophy club than a computer lab.”
RESEARCH
-
OpenAI CEO warns about ChatGPT risks
By
–
When the CEO of OpenAI tells you to worry about Chat GPT it's time to worry ! #ai #artificalintelligence #MachineLearning
#deeplearning #ChatGPT #OpenAI https://
bbc.co.uk/news/world-us-
canada-65616866
… -
Dynamic Planning Advances for Human-Assistant Conversations
By
–
Read all about our recent advances in dynamic planning for human-to-assistant conversations, in which we enable an assistant to plan a multi-turn conversation toward a goal and adapt that plan in real-time via a #ReinforcementLearning-based approach ↓
-
Generative AI for Science: Public Talk and Live Discussion
By
–
Join me, @demishassabis @drfeifei for a public talk and discussion on #GenerativeAI for science this Friday, live-streamed from @WHOSTP website. I will be showcasing the cutting-edge work my teams at @Caltech @nvidia are doing in #ai for science https://
whitehouse.gov/pcast/meetings
/2023-meetings/
… -
GPT-4 Early AGI Version Capabilities According Lex Fridman
By
–
In the Lex Fridman interview, he seemed to think GPT-4 may be a very early version of AGI. So if there are capabilities they are holding back (which is very likely), it’s possible he believes they are way closer than most would know.
-
LLMs and Human Language: A Profound Insight from Stanford Research
By
–
Clever @google
/
@stanford paper on LLMs from my brother @jerryweiAI
. Performance boosts are great, but there is a more profound insight in this paper that was not explicitly stated: LLMs are trained on human language, but due to the nature of how language was developed (first -
Together Compute Raises $20M to Overcome Transformer Limitations
By
–
I spoke with @togethercompute which is exploring some new techniques to get past the Transformer wall. Together raised a $20m seed from Lux Capital and is also responsible for the RedPajamas open source dataset replicating LLaMA’s set.
-
MegaByte: Meta’s Latest Context Window Expansion Breakthrough
By
–
Finally a third one that is super recent is MegaByte, a paper coming out of Meta last week. This one has been getting a lot of buzz. The three approaches are attempts to expand the context window, which allows for more information coming in (and broader hidden prompts).
-
Sub-Quadratic Attention Techniques: FlashAttention and Hyena Advances
By
–
There are a few new techniques under review right now that try to get sub-quadratic attention. The most common one I hear about today is FlashAttention, which evolves the technique to increase the context window. Hyena out of Stanford’s Hazy lab explores convolutional approaches.
-
Context Window Limitations Restrict Advanced AI Model Capabilities
By
–
Some of that has to do with restrictions around context windows, which govern how much information can go into and come out of a prompt. Right now those windows generally fall in the 4k to 8k token range. Already that is showing signs of being insufficient for advanced uses.