2/ Extending Context Window of LLMs – extends the context window of LLMs like LLaMA to up to 32K with minimal fine-tuning (within 1000 steps); attains good performance on several tasks while being more efficient and cost-effective.
Extending LLM Context Windows to 32K with Minimal Fine-tuning
By
–
