That doesn't work for anything with non-persistent buffers (which is most current LLMs AFAICT, since the sin/cos cache is in those buffers)
LLM Buffer Limitations: Non-Persistent Memory Architecture Challenges
By
–
By
–
That doesn't work for anything with non-persistent buffers (which is most current LLMs AFAICT, since the sin/cos cache is in those buffers)