Yup. Other interesting tidbit: to run full finetuning without Adapter on 7B requires at least 8x A100s for comparison.
@rasbt
-
Largest Model Currently Available and Future 120B Model Rumors
By
–
That would be 40B. As far as I know, that’s currently the largest one. A 120B one is in the works though according to rumors.
-
Single Epoch Training on 52k Alpaca Dataset
By
–
This was for one iteration/epoch over the 52k Alpaca dataset.
-
LLaMA-Adapter Works with Any LLM, Including Falcon 40B
By
–
Fun fact: LLaMA-Adapter is not LLaMA-specific. You can use it to finetune any LLM.
— Sebastian Raschka (@rasbt) 8 juin 2023
Below, I finetuned a 40B Falcon model using LLaMA-Adapter (it also works on a single GPU with 20 GB RAM).
Have been heads down tinkering & will follow up with more benchmarks! https://t.co/2Fr6faChv2Fun fact: LLaMA-Adapter is not LLaMA-specific. You can use it to finetune any LLM. Below, I finetuned a 40B Falcon model using LLaMA-Adapter (it also works on a single GPU with 20 GB RAM). Have been heads down tinkering & will follow up with more benchmarks!
-
Video-LLaMA Paper Lacks Quantitative Evaluation Metrics
By
–
"Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Understanding". This method is pretty cool, but what is this new trend where papers don't contain any type of quantitative model evaluation? Looks like researchers are in a rush. https://
arxiv.org/abs/2306.02858 -
Withholding Code While Publishing Implementation Details in Papers
By
–
Similar to "We don't share the code but the details are described in the paper for those who want to implement and use it"
-
Impressive Read on Stable Diffusion and Stability AI
By
–
Oof. What a read! https://
forbes.com/sites/kenrickc
ai/2023/06/04/stable-diffusion-emad-mostaque-stability-ai-exaggeration/
… -
Stability AI Paper Controversy and Latent Diffusion Discussion
By
–
Saw the discussions regarding Stability AI today, & I am a bit surprised that the paper behind it was allegedly swept under the rug.
I do think it was known that it was based on Latent Diffusion? Covered it back in 2022 in an early Ahead of AI issue: https://
magazine.sebastianraschka.com/p/ahead-of-ai-
4-a-big-year-for-ai
… -
GPU Memory Capacity vs Mixed-Precision Training Support Trade-offs
By
–
Hah, yes, same. But it's basically 1 GPU with multiple cores. I am more excited about the 192 Gb GPU memory here. But then, would it support mixed-precision training and bfloat 16? Probably no. So that's actually worse than than an A100 with 48 Gb for example.
-
CORN Method for Finetuning LLMs on Prediction Tasks
By
–
Oh, and I should have emphasized that you can use our CORN (Conditional Ordinal Regression for Neural Networks) method to finetune your favorite LLM for prediction tasks, or course! An example for TripAdvisor customer reviews here: https://
github.com/Raschka-resear
ch-group/coral-pytorch/blob/main/docs/tutorials/pytorch_lightning/distilbert-corn-tripadvisor.ipynb
…