It's just training a very small LoRA you can merge back into the base model, so it should retain the context window!
LoRA Training Preserves Base Model Context Window
By
–
By
–
It's just training a very small LoRA you can merge back into the base model, so it should retain the context window!