I am not aware of a guideline, but it usually also depends on what you are after: (pre)training, finetuning, inference. We have a unified codebase for several LLMs you can try here: https://
github.com/Lightning-AI/l
it-gpt
…

→ View original post on X — @rasbt,