Please see, for example, a paper I worked on to provide more data: https://
arxiv.org/pdf/2104.10350 (and in particular Appendices C and D). (And all the information needed to get things correct as shown in Appendix C was public already). I find the continued use of data now known to be
@jeffdean
-
Scientific Data Integrity in AI Research Papers
By
–
-
Interpreting AI Model Training Costs from Research Paper
By
–
I don't think it's absurd to think that from the paper. If I look at Table 1 from https://
arxiv.org/pdf/1906.02243 paper, it sure seems like the cost of the last row of the table is the cost of "Training one model (GPU)". How else should one interpret that line? Not to mention that -
Neural Architecture Search Methodology Error and Measurement Critique
By
–
Knowing that Neural Architecture Search is a one-time cost per problem domain, not a per-problem cost was definitely widespread knowledge in the ML community at the time of the publication. So that 3261X error was avoidable by simply using their own actual measurements of the
-
Training Cost Estimation Errors in Large Language Models
By
–
I guess this is where we disagree about whether an error of 3261X in estimating the cost of training a language model, or conflating a completely different one-time task to perform a neural architecture search (and overestimating that cost by 88X) to find more energy efficient
-
Updating Papers with Correct Information Benefits the Community
By
–
I'm absolutely not suggesting you retract the paper. I think it's a very good paper in many ways and got people thinking about these issues, which is quite valuable. I just think updating the paper with correct information seems very valuable for the community, since the
-
Environmental impacts of AI: keeping discussion civil
By
–
I'd prefer if we kept the discussion around the actual topic of environmental impacts of AI. Please don't introduce incivility into a civilized discussion.
-
AI Environmental Claims: Misinformation and Carbon Footprint Accuracy
By
–
But you're also aware that the "lifetime of five cars" number is wrong by orders of magnitude. Yet, just last month, you're a co-author on a blog post that continues to propagate this misinformation. https://
huggingface.co/blog/sasha/ai-
environment-primer
… -
Environmental costs of AI research and data integrity issues
By
–
I'm all for that. However that single number is still clouding people's assessment of the environmental costs (e.g. see how this thread started), and @strubell has flatly refused to amend the Arxiv or ACL paper with correct data, and has even co-authored papers after knowing
-
Strubell Study Flawed: Training Cost Estimates Off by 88X
By
–
Did you get this number from Strubell et. al? Because that number was a very flawed estimate that turned out to be off by 88X, and the paper also portrayed it as an every time cost when in fact it was a one time cost. The actual cost of training a model of the size they were
-
Company Team Size and Hiring Mix Strategy
By
–
I think I was the only employee not in the photo, so 35-40. Hiring was a mix: lots of people straight out of grad school or undergrad, but also some people from other companies.