It turns out you can achieve pretty decent performance with far less memory! All you need is this clever new primitive called Cartridge—a pretrained key-value (KV) cache designed to replace the expensive runtime memory footprint of in-context learning (ICL) for large corpora

→ View original post on X — @jiqizhixin