Speeding up LLaMa inference end-to-end by 1.33x on A6000 (for 13B model) and 1.91x on A100 (for 34b model). https://
arxiv.org/abs/2308.16369
LLaMa Inference Speedup 1.33x to 1.91x on GPUs
By
–
By
–
Speeding up LLaMa inference end-to-end by 1.33x on A6000 (for 13B model) and 1.91x on A100 (for 34b model). https://
arxiv.org/abs/2308.16369