4/ LLaMA Pro – proposes a post-pretraining method to improve an LLM’s knowledge without catastrophic forgetting; it achieves this by tuning expanded identity blocks using only new corpus while freezing the inherited blocks.
LLaMA Pro: Post-Pretraining Method Without Catastrophic Forgetting
By
–
