“The demand for energy to power AI models will be infinite.” https://t.co/OxoZGRiONh
— Marek Rosa | European🇪🇺 | South African🇿🇦 (@marek_rosa) 6 octobre 2024
“The demand for energy to power AI models will be infinite.”
RESEARCH
-
AI Models’ Infinite Energy Demand Challenge
By
–
-
Intense Mathematics Competition Sparks Engagement
By
–
Intense math battle!
— Amitav Bhattacharjee (@bamitav) 6 octobre 2024
pic.twitter.com/DGnt30hAG0#Mathematics #math
@KirkDBorne @patricegorissen @jeffkagan @EdwardKens50830 @enilev @insom_ai333 @ChrisCCrowley @sallyeaves @gurmeet_judge @CurieuxExplorer @LavaletteAstrid @bamitav @chidambara09 @adamsconsulting @AkwyZ @alvinfoo…Intense math battle! #Mathematics #math @KirkDBorne @patricegorissen @jeffkagan @radres @enilev @insom_ai333 @cc_crowley @sallyeaves @gurmeet_judge @CurieuxExplorer @LavaletteAstrid @bamitav @chidambara09 @adamsconsulting @AkwyZ @alvinfoo
-
Weights Sharding Strategy: Full Replication Except Last Dimension
By
–
Also, tip on weights sharding: as a generic rule of thumb you can use full replication for all variable dimensions except the last one, which should be sharded across the "model" axis
-
JAX and Keras offer superior large-scale model training performance
By
–
In general, you will find that JAX (and in particular JAX + Keras for the super simple API) offers an exceptionally smooth and performant large-scale model training experience. Zero hassle and high utilization out of the box — there is no alternative that comes close to it.
-
ModelParallel Scaling: Multi-Host Training Setup and Seeding Requirements
By
–
The ModelParallel scheme above scales to arbitrary model sizes and numbers of devices on a single host, but it also works with multi-host training. If you do multi-host training, a common gotcha is that you need to remember to seed everything in the same way on each host (else
-
Can LLMs Learn Visual Understanding Without Images?
By
–
I don’t think you can learn to understand the visual world from text alone. but congenitally blind people learn a ton about the world without visual input, with much richer representations than I would ascribe to LLMs. Landau & Gleitman had a terrific book about this in the
-
Humans trained AI on tiny fraction of internet
By
–
I am sure there is. Brilliant humans have trained on a tiny fraction of the internet.
-
Tesla FSD Safety Data Comparison Flawed Methodology Concerns
By
–
the data below are apples and oranges, because few miles driven by Tesla FSD are without humans in the loop. it’s just not a fair comparison. there are many other problems too (eg which roads, whether conditions, comparing accident rates in much cheaper and lighter cars with
-
Policy and Technical Arguments on AI Safety from Expert Consensus
By
–
More or less. You would expect that in this reality there was a published policy with technical justifications and that the arguments (not just the conclusions) were agreed-upon by technically literate folk such as Geoffrey Hinton.
-
Strubell Study Flawed: Training Cost Estimates Off by 88X
By
–
Did you get this number from Strubell et. al? Because that number was a very flawed estimate that turned out to be off by 88X, and the paper also portrayed it as an every time cost when in fact it was a one time cost. The actual cost of training a model of the size they were