3) VeRA – In LoRA, low-rank matrices A and B are unique for each layer.
– In VeRA, A and B are frozen, random, and shared across all layers.
– Instead, it learns layer-specific scaling VECTORS (b and d) instead. Check this
CODE
-
VeRA: Frozen Shared Matrices with Layer-Specific Scaling Vectors
By
–
-
LoRA: Efficient Fine-tuning with Low-Rank Matrices
By
–
1) LoRA – Add two low-rank trainable matrices, A and B, alongside weight matrices.
– Instead of fine-tuning W, adjust the updates in these low-rank matrices. Even for the largest of LLMs, LoRA matrices take up a few MBs of memory. Check this -
Understanding Matrix Rank for Finetuning Techniques
By
–
Now that we have a basic understanding of rank of a matrix, we're in a good position to understand the different finetuning techniques
-
LLM Weights and Lower-Rank Adaptation in PEFT Techniques
By
–
Some background! LLM weights are matrices of numbers adjusted during finetuning. Most PEFT techniques involve finding a lower-rank adaptation of these matrices—a smaller-dimensional matrix that can still represent the information stored in the original.
-
Top 5 Parameter-Efficient Fine-Tuning Techniques for LLMs
By
–
Traditional fine‑tuning is impractical for LLMs (billions of params; 100s GB). Since this kind of computing isn't accessible to everyone, parameter-efficient finetuning (PEFT) came into existence. Today, we’ll cover the top 5 PEFT techniques, step by step.
-
Top 5 LLM Fine-Tuning Techniques Explained
By
–
I have been fine-tuning LLMs for more that 2 years now! Here are the top 5 LLM fine-tuning techniques, explained with visuals:
-
WordPress categories related to AI topics
By
–
Admittedly not the best example—just the first that popped up in search. Here’s one from less than a month ago, so presumably using 4o or better:
-
Break Down Big Problems Into Solvable Building Blocks
By
–
Take your big problem, break it down into smaller problems, then solve each one using these building blocks chained together.
-
AI Agents as Workflows: DAGs and Code-First Architecture
By
–
AI agents are simply workflows – directed acyclic graphs (DAGs) if you're being precise, or graphs if you include loops. Most steps in these workflows should be regular code – not LLM calls.
-
Seven Core Building Blocks to Solve Business Problems
By
–
You only need about seven core building blocks to solve almost any business problem with AI. 1. Intelligence (LLM)
2. Memory
3. Tools
4. Validation
5. Control
6. Recovery
7. Feedback