Good question! On average: LoRA: 21.33 tokens/sec; Memory used: 14.59 GB
Adapter: 26.22 tokens/sec; Memory used: 14.59 GB
CODE
-
LoRA vs Adapter: Performance and Memory Usage Comparison
By
–
-
Memory reduction: Full finetuning vs Adapter vs LoRA comparison
By
–
Btw, it's also a big reduction in memory: 16 gigs instead of 6 x 40 GB. Due to the reduced parameter counts in the backward pass: Full finetuning: 7,217,189,760
Adapter: 1,365,330
Adapter v2: 3,839,186
LoRA: 3,506,176 (I used LoRA rank of 16 to match Adapter v2 above.) -
GPT Engineer Setup Guide and Demo
By
–
GPT Engineer is pretty crazy… Here's how you set it up: https://t.co/DhAPQSE14i pic.twitter.com/d5N9ITvaYJ
— Dave Ebbelaar (@daveebbelaar) 15 juin 2023GPT Engineer is pretty crazy… Here's how you set it up: https://
youtu.be/gWy-pJ2ofEM -
Efficient Falcon Finetuning with LoRA on Single GPU
By
–
Have been heads-down porting LoRA (low-rank-adaptation) to finetune Falcon more efficiently and ran some performance benchmarks. My longer write-up here: https://
lightning.ai/pages/communit
y/finetuning-falcon-efficiently/
… Long story short: you can finetune Falcon in 1 h on a 52k dataset using a single GPU with 16 GB RAM. -
LLaMA-Adapter Compatible with Falcon Models
By
–
(And yes, that's not a typo, you can use LLaMA-Adapter for Falcon :P)
-
Parameter-Efficient Fine-Tuning of Pretrained Large Language Models
By
–
Now, if we talk about parameter-efficient finetuning of a pretrained LLM, that's a different story.
-
CUDA Deterministic Mode Limitations Across Different Hardware
By
–
I think this is not possible because of CUDA requirements; you can force deterministic mode but in my experience results will still differ across hardware.
-
RLHF Training Method Reduces Human Rater Involvement Requirements
By
–
Yes and afaik the way they use RLHF requires less involvement from human raters, will try to find something
-
Understanding Deep Learning – Comprehensive Neural Network Textbook Released
By
–
Understanding Deep Learning – BOOK Wow, this is such a great deep learning textbook. Covers almost all fundamental neural network techniques and algorithms. The content outline is really nice. Get the book draft: https://
github.com/udlbook/udlboo
k/releases/download/v1.0.4/UnderstandingDeepLearning_08_05_23_C.pdf
… Book website: https://
udlbook.github.io/udlbook/ -
Vercel Accelerator: Building the Future of Development
By
–
We're excited to be part of the Accelerator and see what people build @vercel