4/ LongLLaMA – employs a contrastive training process to enhance the structure of the (key, value) space to extend context length; presents a fine-tuned model that lengthens context and demonstrates improvements in long context tasks.

→ View original post on X — @dair_ai