Even fine tuning regular lora with adafactor would do for now though. With activation checkpointing that’ll handle a reasonable size model
Fine-tuning LoRA with Adafactor and activation checkpointing
By
–
By
–
Even fine tuning regular lora with adafactor would do for now though. With activation checkpointing that’ll handle a reasonable size model