Thanks! Right now, it's a bit inconclusive. Lots of papers show that supervised instruction-finetuning is sufficient (vs RLHF instruction-finetuning). (I cannot put my finger on it, but I feel like we don't have good evaluations, yet, and this is not the end of the story.)
@rasbt
-
QLoRA, Prefix Tuning, and LoRA: Fine-tuning Techniques Comparison
By
–
This was already a lot of coding. I am saving QLoRA for another day! @Yampeleg Regarding prefix tuning/LLaMA-Adapter vs LoRA. The performance is similar, but one advantage of the former is that it allows multimodal inputs (but that's also a post for another day :P)
-
Language Inclusion in Pretraining and Dataset Swapping
By
–
If the language has been included during pretraining, I that shouldn't be a problem (via swapping the dataset). Otherwise, I am not sure:
-
Fine-tuning Limitations and Cross-lingual Transfer in Language Models
By
–
Interesting point! Next to instruction-finetuning, finetuning is usually for a specific task (e.g., language translation). Not sure how well the model would do if you change source language. I.e., I haven't seen any studies investigating that, yet.
-
LoRA-trained models show competitive performance with less overfitting
By
–
The performance of LoRA-trained models is usually pretty competitive. Sometimes even better than fully finetuned (prob due to less overfitting).
(Table from https://
arxiv.org/abs/2106.09685) -
Using 8 GPUs to reduce processing time to minutes
By
–
yes, you can of course use 8 GPUs to bring this down to a few minutes! 🙂
-
LLaMA-Adapter v2 Adds Trainable RMSNorm and Bias Parameters
By
–
Good question. Yes, because LLaMA-Adapter v2 is basically LLaMA-Adapter but it also has trainable RMSNorm parameters and trainable bias units (in FC layers)
-
LoRA vs Adapter: Performance and Memory Usage Comparison
By
–
Good question! On average: LoRA: 21.33 tokens/sec; Memory used: 14.59 GB
Adapter: 26.22 tokens/sec; Memory used: 14.59 GB -
Memory reduction: Full finetuning vs Adapter vs LoRA comparison
By
–
Btw, it's also a big reduction in memory: 16 gigs instead of 6 x 40 GB. Due to the reduced parameter counts in the backward pass: Full finetuning: 7,217,189,760
Adapter: 1,365,330
Adapter v2: 3,839,186
LoRA: 3,506,176 (I used LoRA rank of 16 to match Adapter v2 above.) -
Efficient Falcon Finetuning with LoRA on Single GPU
By
–
Have been heads-down porting LoRA (low-rank-adaptation) to finetune Falcon more efficiently and ran some performance benchmarks. My longer write-up here: https://
lightning.ai/pages/communit
y/finetuning-falcon-efficiently/
… Long story short: you can finetune Falcon in 1 h on a 52k dataset using a single GPU with 16 GB RAM.