Don't take this as hard facts but, from my understanding, the 2B distilled version can be run on as little as 8GB VRAM… Most modern NVIDIA GPUs. The full 13B parameter model would likely need at least a 4090 or 5090.
By
–
Don't take this as hard facts but, from my understanding, the 2B distilled version can be run on as little as 8GB VRAM… Most modern NVIDIA GPUs. The full 13B parameter model would likely need at least a 4090 or 5090.