To make this reproducible on rigs where there are multiple inner steps per training step, isn't the division of the gradient by that number of steps missing?
RESEARCH
-
MiniMax-01: Scaling Foundation Models with Lightning Attention
By
–
introducing MiniMax-01: Scaling Foundation Models with Lightning Attention https://
github.com/MiniMax-AI Hugging Face:
https://
huggingface.co/MiniMaxAI https://
filecdn.minimax.chat/_Arxiv_MiniMax
_01_Report.pdf
… https://
hailuo.ai API:
https://
minimaxi.com/platform -

Transformer²: Self-Adaptive LLMs for Real-Time Task Adaptation
By
–
introducing Transformer²: SELF-ADAPTIVE LLMS https://
arxiv.org/pdf/2501.06252 a novel self-adaptation framework that adapts LLMs for unseen tasks in real-time by selectively adjusting only the singular components of their weight matrices. -
Team praised for exceptional AI model development
By
–
y'all did an amazing work with this model, we're honored!
-

Transformer²: Self-adaptive LLMs that Blur Pre and Post-training
By
–
Transformer²: Self-adaptive LLMshttps://t.co/VnvzSmeIad
— hardmaru (@hardmaru) 15 janvier 2025
This new paper from @SakanaAILabs shows the power of an LLM that can self-adapt its weights to its environment. I think in the future, the line between “pre-training” and “post-training” will be gone, and our models and… https://t.co/t8gEsXOUeg pic.twitter.com/XNFqdnef14Transformer²: Self-adaptive LLMs https://
arxiv.org/abs/2501.06252 This new paper from @SakanaAILabs shows the power of an LLM that can self-adapt its weights to its environment. I think in the future, the line between “pre-training” and “post-training” will be gone, and our models and -
GSK addresses AI hallucinations in drug development
By
–
Hallucinations in #AI: How GSK is addressing a critical problem in drug development https://
venturebeat.com/ai/hallucinati
ons-in-ai-how-gsk-is-addressing-a-critical-problem-in-drug-development/
… via @VentureBeat -
Transformer² System Dynamically Adjusts Weights for Multiple Tasks
By
–
We’re excited to introduce Transformer², a machine learning system that dynamically adjusts its weights for various tasks!https://t.co/ci028qPUWt
— Sakana AI (@SakanaAILabs) 15 janvier 2025
Adaptation is a remarkable natural phenomenon, like how the octopus can blend in with its environment, or how the brain rewires… pic.twitter.com/GLODSaZnu4We’re excited to introduce Transformer², a machine learning system that dynamically adjusts its weights for various tasks! https://
sakana.ai/transformer-sq
uared
… Adaptation is a remarkable natural phenomenon, like how the octopus can blend in with its environment, or how the brain rewires -

Meta Chain-of-Thought: Teaching LLMs to Reason and Think
By
–
Reasoning in #LLMs – Learning How to Think With Meta Chain-of-Thought: https://
arxiv.org/abs/2501.04682 -
Ethical Dilemmas of Artificial Intelligence and Machine Learning
By
–
The ethical dilemmas of artificial intelligence
#AI #AIio #BigData #ML #NLU #Futureofwork @DeepLearn007 @albertogaruccio @alvinfoo @andi_staub @andy_lucerne @antgrasso @BernardMarr @enricomolinari http://
ow.ly/4yR430sJarB -
AI Climate Simulator Explores Geoengineering Solutions
By
–
Just released: New AI Climate Simulator that you can play with. Visualize how geoengineering can slow global warming.
— Andrew Ng (@AndrewYNg) 14 janvier 2025
There is no longer any path to limiting warming to 1.5 degrees Celsius (Paris Agreement), unless we use geoengineering. Reflecting 1% of sunlight away from… pic.twitter.com/KiSm83e3RFJust released: New AI Climate Simulator that you can play with. Visualize how geoengineering can slow global warming. There is no longer any path to limiting warming to 1.5 degrees Celsius (Paris Agreement), unless we use geoengineering. Reflecting 1% of sunlight away from