I think we are talking about the same thing. That would only work after training. It's based on the assumption that the matrices have low rank for a specific target task but are not low rank in general (for the pretraining tasks).

→ View original post on X — @rasbt