LLMS

Hugging Face PEFT Project: Open Source Repository and Blog Overview

By

–

15 February 2023 9h30

This Project from Hugging Face is completely Open Source, Check out the repo here: https://
github.com/huggingface/pe
ft
… Consider checking this blog from Hugging Face Team for more detailed overview. https://
huggingface.co/blog/peft 5/5

Getting Started with PEFT: Simple 3-Step Setup Guide

By

–

15 February 2023 9h30

Finally Getting started with PEFT is really simple: 1. Import the Necessary Libraries
2. Define the Config with PEFT method
3. Wrapping base model from Hugging Face Transformers by calling `get_peft_model` That's it. You can start training now. Check the below code 4/5

Interesting Use Cases of PEFT Parameter Efficient Fine-Tuning

By

–

15 February 2023 9h30

Some of the interesting usecases of PEFT are: 1. Stable Diffusion Dreambooth training.
2. Finetuning the `bigscience/T0_3B` model which has around 3 Billion Parameters. Checkout them here: https://
github.com/huggingface/pe
ft#use-cases
… 3/5

PEFT: Fine-tuning Large Models on Low-End Hardware

By

–

15 February 2023 9h30

But fine-tuning large models on low-end hardware is a real challenge PEFT solves this by fine-tuning a small number of model parameters while freezing most parameters of the pre-trained LLMs. This reduces the computational and storage costs 2/5 https://
github.com/huggingface/pe
ft
…

Fine-tuning: Faster Model Adaptation for Specific Tasks

By

–

15 February 2023 9h30

In simple words fine-tuning is taking a pre-trained machine learning model and adjusting it for a specific task Fine-tuning is much faster than training a model from scratch and also reduces the amount of data, compute required for training and lot of other benefits 1/5

PEFT: Fine-tune Large Language Models on Low-Resource Hardware

By

–

15 February 2023 9h30

Large Language models are expensive to fine-tune and it's really difficult to train them on local hardware. Introducing PEFT ("Parameter-Efficient Fine-Tuning") from Hugging Face that lets you fine-tune Billion Parameter Models on Low-Resource Hardware. A Thread