Depends on your settings. But if you limit the context size to like 2048 (like in the NeurIPS competition) and use a microbatch size of 1 with gradient accumulation and qlora with llama 2 7B, that’s approx 20 GB RAM and shouldn’t take too long, maybe an hour.

→ View original post on X — @rasbt,