5/ Efficient Context Window Extension of LLMs – proposes a compute-efficient method for efficiently extending the context window of LLMs beyond what it was pretrained on; models of up to 128K context length reproduced.
Efficient Context Window Extension Method for Large Language Models
By
–
