two years ago the entry point to participate in AI research was to have a couple of GPUs for training or finetuning now the entry level is to be able to regularly train a 50-70B params model on a couple hundred billion tokens

→ View original post on X — @thom_wolf,