That said, CAG has downsides:
• You need lots of memory as you fill the context window at every query.
• You cannot upload your whole million-rows dataset. It must be smaller than the current model's context window (e.g. 128k, 200k… tokens).
• You must host the model

→ View original post on X — @whats_ai