Update: Recollection of the start of the conversation reliably fails after 7 repetitions of the ~1024-token input. This suggests that text-davinci-002-render, the model underlying ChatGPT, has a likely context length of 8192 tokens, with some tokens consumed by formatting.
RESEARCH
-
ChatGPT recalls content beyond 4096-token context window
By
–
A demonstration that ChatGPT is able to recall the content of conversations beyond the length of a 4096-token context window. Uses four messages of 1024+ tokens, each ending with instructions to demonstrate that input is not truncated before the model sees it.
-
Falcon: Model-Based RL for Extreme Turbulent Environments
By
–
— Prof. Anima Anandkumar (@AnimaAnandkumar) 3 décembre 2022
Tune in for our talk on Falcon, first model-based RL method in real-world extreme turbulent environments (Reynolds Num. 230000!), AI4Science workshop @ #Neurips2022, today @ 1:20pm. Falcon achieves 37% improvement over SOTA in stabilization of an airfoil using only 8 min of data!
-
Interview on Rule of the Robots and AI Future
By
–
My interview on #RuleoftheRobots and the future of #AI with the Korean technology news outlet @TheMiilkInc
. -
Celebrating Conversational AI Pioneers and Their Legacy
By
–
btw, can we spend some time to congratulate the OGs scientists of conversational AI like @jaseweston @AlecRad
, and many others? I still remember fondly our convAI days at Neurips 19, the field has gone a long way since then thanks to many many people! -
Python Libraries: Essential Tools for Data Scientists
By
–
#Python Libraries are collections of functions and methods that allows #DataScientist to perform many actions without writing #code https://
buff.ly/2QksUva Via @ingliguori #MachineLearning #BigData #DataScience #ArtificialIntelligence #cybersecurity #DX #Analytics #AI -
Adjacent Impossible and AGI-Hard: Categorizing AI Progress
By
–
There's two lists to maintain – i've been calling it the "Adjacent Impossible" (stuff that it'll probably do soon) and "AGI-hard" (after @tszzl
) https://
lspace.swyx.io/p/agi-hard -
Sparsity Research Reduces AI Training and Inference Costs
By
–
Our team was at #NeurIPS this week, where we made several announcements regarding our research into sparsity. Read our overview of sparsity and how it can be used to reduce cost and time of training and inference: https://
hubs.li/Q01tHYvr0 -
Montreal.AI: Global Leader in Reinforcement Learning and AI Agents
By
–
The fact is that http://
MONTREAL.AI is a worldwide leader in #ReinforcementLearning and #AIAgents since at least 2015 . . . #MontrealAI -
Elon Musk announces Neuralink brain chip implant upcoming
By
–
[FLASHER] @ElonMusk annonce qu'il s'implantera bientôt une puce #Neuralink https://
buff.ly/3gSA4rG v/ @CNBC L'implant cérébral, entouré de scepticisme, est attendu sous 6 mois et doit restituer la vue aux aveugles et la motricité aux paralysés #Tech #FlashTweet