Theoretically, you can get it to work if you use deepspeed stage 3 with offloading. But I see there is currently a little bug for stage 3 and LoRA. I think we watned to switch from DeepSpeed to FSDP anyway though. PS: Glad you like my book!
@rasbt
-
Deep Learning Course Completion Badge and Final Exam Released
By
–
Am so glad about all the nice feedback about my Deep Learning course — happy to hear that it was useful to tens of thousands of people! Just added a final exam & completion badge for LinkedIn! So, if you'd like to test your knowledge with 25 questions: https://
lightning.ai/pages/ai-educa
tion/deep-learning-fundamentals/certification/
… -
Agreement on Twitter Algorithm Open Source Release
By
–
Totally agree! https://
github.com/twitter/the-al
gorithm
… -
Custom LLMs Essential for Specialized Domain Applications
By
–
If you want to develop/use a generalist chatbot, I totally agree. But there are countless problems where a custom LLM makes 100% sense. From spam classification to working with financial, law or medical data to protein sequence modeling.
-
Batch Size Analogy for Precision Selection in Hardware
By
–
It makes sense though since it’s kind of like batch size: make it as large as your hardware permits (… < bfloat 16 < bfloat 16/32 mixed < float 32)
-
Clarification on supervised instruction-finetuning versus RLHF methods
By
–
Btw. was supervised instruction-finetuning vs RLHF instruction-finetuning what you had in mind, or were you more curious about supervised finetuning as in classification vs supervised instruction-finetuning?
-
Instruction-Finetuning vs RLHF: Inconclusive Research Findings
By
–
Thanks! Right now, it's a bit inconclusive. Lots of papers show that supervised instruction-finetuning is sufficient (vs RLHF instruction-finetuning). (I cannot put my finger on it, but I feel like we don't have good evaluations, yet, and this is not the end of the story.)
-
QLoRA, Prefix Tuning, and LoRA: Fine-tuning Techniques Comparison
By
–
This was already a lot of coding. I am saving QLoRA for another day! @Yampeleg Regarding prefix tuning/LLaMA-Adapter vs LoRA. The performance is similar, but one advantage of the former is that it allows multimodal inputs (but that's also a post for another day :P)
-
Language Inclusion in Pretraining and Dataset Swapping
By
–
If the language has been included during pretraining, I that shouldn't be a problem (via swapping the dataset). Otherwise, I am not sure:
-
Fine-tuning Limitations and Cross-lingual Transfer in Language Models
By
–
Interesting point! Next to instruction-finetuning, finetuning is usually for a specific task (e.g., language translation). Not sure how well the model would do if you change source language. I.e., I haven't seen any studies investigating that, yet.