Ah sorry, I meant M1/M2 chips (not specifically M1/2 CPUs). As far as I know, the 4-bit NormalFloat format that is used in QLoRA is currently only supported on Nvidia GPUs (
https://
github.com/TimDettmers/bi
tsandbytes/issues/485
…). Maybe the repo you mentioned uses a different type of quantized training.
@rasbt
-
QLoRA 4-bit NormalFloat format supported only on Nvidia GPUs
By
–
-
LoRA Compared to Llama-Adapter and Llama-Adapter v2
By
–
LoRA is a parameter-efficient finetuning technique, yes. I recently compared to Llama-Adapter and Llama-Adapter v2:
-
Alternative Precision Flag for Non-GPU Machines
By
–
I haven't tried on non-GPU machines, but maybe the following works:
"–precision 16-true" instead of "–precision bf16-true" -
BFloat16 Support in Ampere GPUs and M1 Chips
By
–
BFloat16 is a format introduced in Ampere architectures (Nvidia A100 cards) or newer. M1 chips for some reason have also bfloat16 in PyTorch, but I think this MPS bfloat16 is something very different.
-
BFloat16 Weight Clipping Limits and Extreme Value Representation
By
–
Hm, so that means all weights above values 65504 and below -65504 will be clipped. Does anyone know of there are numbers that large/small in the original bfloat16 representation?
-
LoRA as Cost-Effective Solution for Multi-Customer LLM Deployment
By
–
I believe this would be easiest and most cost-effective because they won't have to store a new 175B parameter LLM for each customer then. With LoRA, you can use the same base LLM but only need to store the LoRA weights.
-
Lit-GPT: Lightning AI’s New GitHub Repository Launch
By
–
Lit-GPT 🙂 https://
github.com/Lightning-AI/l
it-gpt
… -
Scaling Llama 2 to GPT-3 Size for Performance Comparison
By
–
Someone needs to scale up Llama 2 to GPT 3 size. Would be curious how that compares then.
-
LoRA Services vs Open-Source: Llama 2 Competitive Comparison
By
–
Will be interesting how this LoRA-on-demand service will compare to open-source LoRA on prem. Here's a little reminder that open-source Llama 2 compares very favorably to ChatGPT / GPT 3.5
-
ML Evaluation Metrics and Lit-GPT Evaluation Framework
By
–
I have a write-up explaining BLEU, ROUGE, and BERTScore in Chapter 19 of my ML Q and AI book: https://
leanpub.com/machine-learni
ng-q-and-ai/
… Other than that, Lit-GPT has currently the Eleuther AI Evaluation Harness implemented: https://
github.com/Lightning-AI/l
it-gpt/blob/main/tutorials/evaluation.md
…