Exactly! Tool outputs are usually long and stick around in context, so the cache grows fast. Prompt caching helps the prefill side, but decode still has to read all of it on every token. Trimming what goes back into context matters more than people realize.
LLMS
-
Every ML conference since 2019 features at least three subquadratic attention papers
By
–
every ML conference I've been to since 2019 has had no fewer than three papers proposing new techniques for subquadratic attention
-
New sub-quadratic attention technique makes long-context LLMs 10x cheaper
By
–
"Introducing a breakthrough new technique for sub-quadratic attention, making long-context LLMs 10x cheaper without sacrificing performance" Me:
-
Zyphra small model impressive but reasoning still inaccurate
By
–
I love Zyphra, and it's amazing how much they can do in a small model but the reasoning is not yet ready (extremely long winded but still inaccurate). I am looking forward to the next update
-
Anthropic releases API for Claude’s memory-organizing Dream feature
By
–
Yesterday, Anthropic just released the API for this Dream, which is really fascinating. What it does is let the Claude agent "dream" on its own, organizing the memories it wrote down in past sessions—merging duplicates, updating expired ones, unifying contradictions, and even
-
Subquadratic Attention and Data Quality: Challenges for Large Context AI Models
By
–
people on here are dumb. the latest subquadratic attention trick might produce a model that *processes* 1M tokens (or 12M..) without going insane, but that doesn't make it good the real problem isn't the architecture, it's the data. humans haven't generated many contiguous
-
Epstein suicide note OCR benchmark test results
By
–
Epstein Suicide Note OCR Benchmark test: 1. Gemini 85%
2. GPT-5.5 82%
3. Grok 4.3 80%
4. Opus 4.7 75% -
Local Grok Model for Robot Task Execution
By
–
I assume there will be a local version of Grok running in the robot. After all telling it to "please clean up this mess" doesn't require a huge model.
