Yes, and the gap is larger than most expect. A 4K system prompt can add hundreds of ms to TTFT before the model says anything. Streaming hides decode latency but does nothing for prefill. Trimming the system prompt usually beats any decode-side optimization people try first.
LLMS
-
Luma Labs Releases Uni-1.1 Multimodal Transformer API
By
–
🚨 @LumaLabsAI just dropped the Uni-1.1 API—and it's already #1 on Human Preference Elo.
— Futurepedia – Learn to Leverage AI (@futurepedia_io) 7 mai 2026
This isn't just another image gen API. It's a unified transformer that reasons across text + image tokens in a single sequence (not bolted-together models).
What you can do: 🧵 👇️… pic.twitter.com/MQSyM3NfTX@LumaLabsAI just dropped the Uni-1.1 API—and it's already #1 on Human Preference Elo. This isn't just another image gen API. It's a unified transformer that reasons across text + image tokens in a single sequence (not bolted-together models). What you can do:
-
Prompt Caching and Context Trimming in AI Models
By
–
Exactly! Tool outputs are usually long and stick around in context, so the cache grows fast. Prompt caching helps the prefill side, but decode still has to read all of it on every token. Trimming what goes back into context matters more than people realize.
-
Every ML conference since 2019 features at least three subquadratic attention papers
By
–
every ML conference I've been to since 2019 has had no fewer than three papers proposing new techniques for subquadratic attention
-
New sub-quadratic attention technique makes long-context LLMs 10x cheaper
By
–
"Introducing a breakthrough new technique for sub-quadratic attention, making long-context LLMs 10x cheaper without sacrificing performance" Me:
-
Zyphra small model impressive but reasoning still inaccurate
By
–
I love Zyphra, and it's amazing how much they can do in a small model but the reasoning is not yet ready (extremely long winded but still inaccurate). I am looking forward to the next update
-
Anthropic releases API for Claude’s memory-organizing Dream feature
By
–
Yesterday, Anthropic just released the API for this Dream, which is really fascinating. What it does is let the Claude agent "dream" on its own, organizing the memories it wrote down in past sessions—merging duplicates, updating expired ones, unifying contradictions, and even
-
Subquadratic Attention and Data Quality: Challenges for Large Context AI Models
By
–
people on here are dumb. the latest subquadratic attention trick might produce a model that *processes* 1M tokens (or 12M..) without going insane, but that doesn't make it good the real problem isn't the architecture, it's the data. humans haven't generated many contiguous
-
Epstein suicide note OCR benchmark test results
By
–
Epstein Suicide Note OCR Benchmark test: 1. Gemini 85%
2. GPT-5.5 82%
3. Grok 4.3 80%
4. Opus 4.7 75%
