In July 2024, we released emissions estimates for pre-training the Gemma 2 model series (2B, 9B, and 27B parameter models, so ~12X, ~43X, and ~128X larger than the 213M parameter models you were looking at, and respectively trained on 2T, 8T, and 13T tokens of data). We estimate

→ View original post on X — @jeffdean