One bug in bitsandbytes caused QLoRA to consume an extra *read notes* 4888 GB of VRAM. Even with QLoRA, fine-tuning Llama 3 405B on long sequence lengths ain't cheap.
QLoRA Bug Consumes 4888GB VRAM in Llama 3 405B
By
–
By
–
One bug in bitsandbytes caused QLoRA to consume an extra *read notes* 4888 GB of VRAM. Even with QLoRA, fine-tuning Llama 3 405B on long sequence lengths ain't cheap.