Just implemented LoRA (
https://
github.com/Lightning-AI/l
it-parrot/pull/128
…) and may be able to tell more soon. (Stay tuned for more info next week. Have to get all my analyses organized and write that up properly before mixing up some numbers there. Probably no qlora, but some info on LoRA at least)

→ View original post on X — @rasbt,