If I had to guess it's less decay and more that memories have naive RAG-like implementations, so you're at the mercy of whatever happens to retrieve in the top k via embeddings. They don't process you in aggregate and over time (probably compute constraints) so they struggle to
RESEARCH
-
The AI Scientist Published in Nature: Fully Automated Research Milestone
By
–
I’m incredibly proud of The AI Scientist team for this milestone publication in @Nature. We started this project to explore if foundation models could execute the entire research lifecycle. Seeing this work validated at this level is a special moment. I truly believe AI will forever change the landscape of how scientific discoveries and scientific progress are made. Sakana AI (@SakanaAILabs) The AI Scientist: Towards Fully Automated AI Research, Now Published in Nature Nature: nature.com/articles/s41586-0… Blog: sakana.ai/ai-scientist-natur… When we first introduced The AI Scientist, we shared an ambitious vision of an agent powered by foundation models capable of executing the entire machine learning research lifecycle. From inventing ideas and writing code to executing experiments and drafting the manuscript, the system demonstrated that end-to-end automation of the scientific process is possible. Soon after, we shared a historic update: the improved AI Scientist-v2 produced the first fully AI-generated paper to pass a rigorous human peer-review process. Today, we are happy to announce that “The AI Scientist: Towards Fully Automated AI Research,” our paper describing all of this work, along with fresh new insights, has been published in @Nature! This Nature publication consolidates these milestones and details the underlying foundation model orchestration. It also introduces our Automated Reviewer, which matches human review judgments and actually exceeds standard inter-human agreement. Crucially, by using this reviewer to grade papers generated by different foundation models, we discovered a clear scaling law of science. As the underlying foundation models improve, the quality of the generated scientific papers increases correspondingly. This implies that as compute costs decrease and model capabilities continue to exponentially increase, future versions of The AI Scientist will be substantially more capable. Building upon our previous open-source releases (github.com/SakanaAI/AI-Scien…), this open-access Nature publication comprehensively details our system's architecture, outlines several new scaling results, and discusses the promise and challenges of AI-generated science. This substantial milestone is the result of a close and fruitful collaboration between researchers at Sakana AI, the University of British Columbia (UBC) and the Vector Institute, and the University of Oxford. Congrats to the team! @_chris_lu_ @cong_ml @RobertTLange @_yutaroyamada @shengranhu @j_foerst @hardmaru @jeffclune — https://nitter.net/SakanaAILabs/status/2036840833690071450#m
→ View original post on X — @_yutaroyamada, 2026-03-25 16:24 UTC
-
LLMs Overfitting to RAG Context: A Systemic Training Bias
By
–
(I cycle through all LLMs over time and all of them seem to do this so it's not any particular implementation but something deeper, e.g. maybe during training, a lot of the information in the context window is relevant to the task, so the LLMs develop a bias to use what is given, then at test time overfit to anything that happens to RAG its way there via a memory feature (?))
-
The AI Scientist Published in Nature: Fully Automated Research
By
–
The AI Scientist: Towards Fully Automated AI Research, Now Published in Nature Nature: nature.com/articles/s41586-0… Blog: sakana.ai/ai-scientist-natur… When we first introduced The AI Scientist, we shared an ambitious vision of an agent powered by foundation models capable of executing the entire machine learning research lifecycle. From inventing ideas and writing code to executing experiments and drafting the manuscript, the system demonstrated that end-to-end automation of the scientific process is possible. Soon after, we shared a historic update: the improved AI Scientist-v2 produced the first fully AI-generated paper to pass a rigorous human peer-review process. Today, we are happy to announce that “The AI Scientist: Towards Fully Automated AI Research,” our paper describing all of this work, along with fresh new insights, has been published in @Nature! This Nature publication consolidates these milestones and details the underlying foundation model orchestration. It also introduces our Automated Reviewer, which matches human review judgments and actually exceeds standard inter-human agreement. Crucially, by using this reviewer to grade papers generated by different foundation models, we discovered a clear scaling law of science. As the underlying foundation models improve, the quality of the generated scientific papers increases correspondingly. This implies that as compute costs decrease and model capabilities continue to exponentially increase, future versions of The AI Scientist will be substantially more capable. Building upon our previous open-source releases (github.com/SakanaAI/AI-Scien…), this open-access Nature publication comprehensively details our system's architecture, outlines several new scaling results, and discusses the promise and challenges of AI-generated science. This substantial milestone is the result of a close and fruitful collaboration between researchers at Sakana AI, the University of British Columbia (UBC) and the Vector Institute, and the University of Oxford. Congrats to the team! @_chris_lu_ @cong_ml @RobertTLange @_yutaroyamada @shengranhu @j_foerst @hardmaru @jeffclune
→ View original post on X — @sakanaailabs, 2026-03-25 16:21 UTC
-
Gemini Deep Think Revolutionizes Scientific Research with Advanced AI
By
–
Gemini Deep Think: Redefining the Future of Scientific Research https://
buff.ly/5K3sIxP
#AI #MachineLearning #DeepLearning #LLMs #DataScience -
The Challenges of Personalization in Language Models
By
–
One common issue with personalization in all LLMs is how distracting memory seems to be for the models. A single question from 2 months ago about some topic can keep coming up as some kind of a deep interest of mine with undue mentions in perpetuity. Some kind of trying too hard. [Translated from EN to English]
-
AI Scientist article heralds new era for scientific discovery
By
–
The AI Scientist, a new @Nature article making the case:
"The dawn of a new era in which the process of discovery is no longer a solely human pursuit and in which the pace at which we are able to reap the harvest of scientific discovery could accelerate dramatically." -
Multi-agent systems need shared definitions prevent miscommunication
By
–
Shared language =/= shared meaning. And that can turn multi-agent systems into a game of telephone without any human in the loop being the wiser. Our @MicrosoftAI pre-print tests a solve: if agents don't agree on a definition, they can't use the term. The results: disagreement
-
Context Engineering: Why Quality Data Beats Volume for AI
By
–
Most teams think “more data = smarter AI.” I make the opposite case: context beats volume. When LLMs are grounded in your company’s own signals—not just the internet—they deliver accurate, explainable decisions at scale. A thread on Context Engineering and why it changes
-
Context Engineering: Structuring Data for AI Reasoning
By
–
What is “Context Engineering”? It’s the discipline of stitching your unstructured mess—logs, chats, docs, images—into something an AI can actually reason over. → Vector DBs + hybrid search + embeddings to retrieve by meaning, not keywords. → Decisions anchored in your data,