4) Delta-LoRA – It tunes the matrix W as well, but not in the traditional way.
– Here, the difference (or delta) between the product of matrices A and B in two consecutive training steps is added to W. Check this
Delta-LoRA: Matrix Tuning Innovation in Model Training
By
–
