Still WIP I would assume that it would raise cloud costs and load on their systems quite a lot. Not clear also how would it play with the context window itself.
Technical trade-offs of LLM inference costs and context window management
By
–
By
–
Still WIP I would assume that it would raise cloud costs and load on their systems quite a lot. Not clear also how would it play with the context window itself.