A counter-argument is that multi-epoch training works pretty well for ViTs.
MACHINE LEARNING
-
Scaling LLM Performance: Token-Crisis Solutions and Diminishing Returns
By
–
They actually might be doing this already but yeah, probably diminishing returns.
Some insights in the "To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis" paper: https://
arxiv.org/abs/2305.13230 -
Why Tech Giants Don’t Train ML Models Longer Despite Low Costs
By
–
It has been pointed out that even $100M ML model training costs aren’t that large in tech titan terms, so I wonder why they don’t just keep training them for far longer just to see what happens. Overfitting is a concern, but you might also hit a grokking discontinuity.
-
JAX Deep Learning API: Keras Backend and Framework Choices
By
–
The problem with JAX always on the deep learning API side. Imho, it needed something like Keras. Nice that it’s now a backend! Btw, do use pure JAX in Keras or one of the intermediate APIs (like Flax)?
-
Emergent Abilities in LLMs: Responding to LeCun’s Critique
By
–
Yann LeCun is obviously a legend but I found this tweet to be quite misinformed. The whole point of "emergent abilities" such as few-shot prompting and chain-of-thought prompting, is that we clearly *did not* explicitly train or fine-tune them into the model. These abilities
-
AI Progress Accelerating: Capabilities, Compute, and Data Surge
By
–
The coming wave IS coming. Progress is not slowing but speeding up. This piece makes that crystal clear. Capabilities, compute, data… Everything is going up. These patterns are deep and relentless. https://
time.com/6300942/ai-pro
gress-charts/
… -
Machine Learning Applications for Personalized Disease Treatment
By
–
Machine Learning for Targeted Disease Treatment
#AI #AIio #BigData #ML #NLU #Futureofwork @DeepLearn007 @albertogaruccio @alvinfoo @andi_staub @andy_lucerne @antgrasso @BernardMarr @enricomolinari http://
ow.ly/MnXL30swzJI -
Open Catalyst Demo Accelerates Computational Chemistry by 1000x
By
–
Last week, we released the Open Catalyst demo. This work enables scientists to simulate the reactivity of catalyst materials ~1000x faster than existing computational methods — and at a scale of catalyst surface-adsorbate combinations impossible to explore without AI.
— AI at Meta (@AIatMeta) 4 août 2023
Demo ⬇️Last week, we released the Open Catalyst demo. This work enables scientists to simulate the reactivity of catalyst materials ~1000x faster than existing computational methods — and at a scale of catalyst surface-adsorbate combinations impossible to explore without AI. Demo
-
Role Of AI In Creating A Sustainable Future
By
–
Role Of AI In Creating A Sustainable Future
#AI #AIio #BigData #ML #NLU #Futureofwork @oluskayacan @richardkimphd @MariaFariello @johnpearcenews5 @ResurgentAV @rohanmuralee @globaliqx http://
ow.ly/6mHi30swzJH -
Cohere Launches Second NLP Scholars Program Cohort
By
–
Our research lab @forai_ml has just launched its second cohort of the Scholars Program. Check out this opportunity to work with some of the best researchers in NLP! Learn more and apply by September 11th: http://
txt.cohere.com/c4ai-scholars-
program/
…