I definitely like the paper in that it focused attention on the potential issue and started a good (& still ongoing) discussion about the right methodology. I just wish it hadn't had flaws that caught people's attention w/eye-catching numbers that turned out not to be correct.
@jeffdean
-
Rock Star Developers at Khipu AI
By
–
Maybe they're "rock star developers"? (of awesome AI talent at @Khipu_AI
!) -
NAS-Generated Evolved Transformer Open Sourced for Community
By
–
Indeed, it already has, and that particular NAS output (the Evolved Transformer) was open sourced for everyone to benefit from.
-
Accurate CO2e Measurements for ML Models Matter
By
–
An important thread about misrepresentation of CO2e numbers for different ML models. I believe it's good to use actual measurements of this data, rather than estimates that are flawed in a variety of ways.
-
Google Cloud Infrastructure and AI Hardware Energy Efficiency Comparison
By
–
All the infrastructure is accessible to external entities through our Google Cloud Platform. Table 1 in https://
arxiv.org/abs/2104.10350 has columns that enables you to compare moving from avg US datacenter/electrical grid to Google datacenter, & also moving from P100 to TPUv2. -
Google’s Flawed Training Cost Estimates Criticized by Researcher
By
–
I have raised this exact point in the past with some Google authors, saying that it wasn't reasonable to use flawed estimates that were off by a factor of 3261X from the actual cost of training this language model, and that they should fix this before publishing. Sigh.
-
Language Model Training Efficiency on Google TPU Hardware
By
–
… and off by ~118,000X if you consider training that language model on a Google TPUv2 in our Iowa datacenter. See Table 1 and Appendix C and D of https://
arxiv.org/abs/2104.10350 for more details. -
Language Model Training Emissions 3261X Lower Than Previous Estimates
By
–
The actual emissions for training a language model of the scale examined by Strubell et al. is 3261X smaller than this flawed estimate even if you consider the "P100 in average US data center on average electrical grid" scenario they were evaluating
… -
NAS Search Emissions Estimate Off by 88X
By
–
… meaning the actual estimate for the one-time NAS search was off by 88X.
This highly flawed estimate was then portayed as the emissions for training a language model, rather than the one-time cost of the NAS. … -
CO2e Estimation Errors in AI Model Training Studies
By
–
…then estimated the CO2e as if it was run on P100 GPUs in average U.S. datacenter (much higher emissions, much worse PUE, so off by ~5X from actual running environment). Worse, they missed the use of smaller proxy tasks by So et al & so were off by a further factor of 19X…