We converted a full-precision (FP16) Llama 3 8B model into a 4‑bit version using Hugging Face’s bitsandbytes library with the “nf4” configuration. This reduced the model’s memory usage from ~15 GB to ~5.4 GB—a savings of roughly 64%.
Llama 3 8B quantized to 4-bit reduces memory usage by 64%
By
–