As an extreme version: taking work that was actually run on EfficientAcceleratorsA in SolarPowerDataCenterB & then estimating what happens if you were to run it on WarehouseOfIBMPCs powered by CoalPowerPlant & saying work produces lots of emissions doesn't seem reasonable to me.
@jeffdean
-
Carbon emissions computation carbon footprint AI model training
By
–
Because carbon emissions is highly dependent on which data center you run in, which electricity grid it is powered by, etc. If you want to talk about the amount of computation needed to train one model architecture vs. another, that seems reasonable to compare abstractly.
… -
Smaller proxy tasks in So et al. Section 3.3 and Appendix C
By
–
The use of smaller proxy tasks was covered in Section 3.3 of So et al., so no Google inside knowledge needed (although it's pretty subtly described in So et al.) There's some discussion in Appendix C of
-
NAS Cost Methodology: Distinguishing One-Time Versus Recurring Expenses
By
–
Presenting the one-time cost of a NAS that has already been conducted in an energy efficient environment (and the best architecture from it open sourced) as if it was an every problem cost is pretty confusing to the reader. "Counting building the factory for every car produced."
-
Reflections on Khipu AI Conference and LatAm Research Community
By
–
I'm sorry I wasn't able to make it to @Khipu_AI this year. I had such an amazing time & met so many great researchers, students & other LatAm AI community members when I attended in 2019. I enjoyed living vicariously through awesome pictures of @Khipu_AI
'23 in my Twitter feed! -
NAS Results Reduce Language Model Training Emissions by 1.3X
By
–
It's actually more ironic. The results of one-time NAS (which took 88X less CO2 than the external paper estimated) are open-sourced and make training language models 1.3X faster & produce 1.3X less emissions (see figure 4 in https://
arxiv.org/abs/2104.10350) https://
github.com/tensorflow/ten
sor2tensor/blob/master/tensor2tensor/models/evolved_transformer.py
… -
Neural Network Search Emissions Metric Accuracy Questioned
By
–
Fwiw, "lifetime emissions of five cars" metric is wrong & based on inaccurate data & confusion about the one-time vs. every problem misunderstanding of neural architecture search (NAS), proxy tasks, &HW used. It's really a factor of 19X, 88X, 3261X, or 118,000X less than that.
-
Similar magnitude argument presented in Section 5 of arXiv paper
By
–
We make a similar magnitude argument in Section 5 of https://
arxiv.org/abs/2204.05149 -
Google ML Energy Use Under 15% Overall 2019-2021
By
–
In follow-on work, we measured the energy use of ML at Google (inference, training, and research) to be <15% of overall energy use in 2019, 2020, and 2021 (section 5 has some details, but more about aggregate energy use rather than individual models). https://
arxiv.org/abs/2204.05149 -
Strubell Paper Emissions Data Accuracy Clarification
By
–
(To be very clear, none of the flaws were intentional, and we have discussed with the authors of the Strubell et al. paper. I do think making people aware of how that data is not actual measured emissions cost and is off by quite a bit is pretty important).