I have raised this exact point in the past with some Google authors, saying that it wasn't reasonable to use flawed estimates that were off by a factor of 3261X from the actual cost of training this language model, and that they should fix this before publishing. Sigh.
LLMS
-
Language Model Training Efficiency on Google TPU Hardware
By
–
… and off by ~118,000X if you consider training that language model on a Google TPUv2 in our Iowa datacenter. See Table 1 and Appendix C and D of https://
arxiv.org/abs/2104.10350 for more details. -
Language Model Training Emissions 3261X Lower Than Previous Estimates
By
–
The actual emissions for training a language model of the scale examined by Strubell et al. is 3261X smaller than this flawed estimate even if you consider the "P100 in average US data center on average electrical grid" scenario they were evaluating
… -
NAS Search Emissions Estimate Off by 88X
By
–
… meaning the actual estimate for the one-time NAS search was off by 88X.
This highly flawed estimate was then portayed as the emissions for training a language model, rather than the one-time cost of the NAS. … -
More Effective Vocabulary Learning Method Using ChatGPT
By
–
a more effective way to learn vocab words than straight memorization with ChatGPT
-
Google’s Universal Speech Model Scales ASR to 100+ Languages
By
–
Google’s Universal Speech Model Scales Automatic Speech Recognition to 100+ Languages https://
syncedreview.com/2023/03/09/goo
gles-universal-speech-model-scales-automatic-speech-recognition-to-100-languages/
… -
Learn LLMs and Build Auto-Summarising Bot in 4 Hours
By
–
Replit is the fastest way to get started with generative AI, and now our friends at @AICampOfficial bring a new course to teach you all about LLMs. In just 4 hours, you'll learn the powers of GPT3 and end up building your own auto-summarising bot. Genius!
-
GPT-4 next week with multimodal models offering video capabilities
By
–
"We will introduce GPT-4 next week, there we will have multimodal models that will offer completely different possibilities – for example videos," Braun said.
-
GPT-4 coming next week, multimodal, says Microsoft Germany CTO
By
–
GPT-4 is coming next week – and it will be multimodal, according to Microsoft Germany CTO Andreas Braun
-
Facebook’s LLaMA AI model leaked onto GitHub
By
–
In case you missed it: Facebook's LLaMA AI model, intended for government and approved researchers only, was leaked onto Github: https://
github.com/facebookresear
ch/llama
…