> Hugging Face releases Picotron, a microscopic lib that solves LLM training 4D parallelization Llama-3.1-405B took 39 million GPU-hours to train, which represents 4.5 thousand years. If they had needed all this time, we would have GPU stories from the time of Pharaoh

→ View original post on X — @aymericroucher