Could you share your traces and the resulting model on HF? Would be super interesting
RESEARCH
-
Sparse Attention and Large Context Windows: An Industry Shift in AI Models
By
–
确实很酷，拿 2900 万美金做 12M context，
— 艾略特 (@elliotchen100) 5 mai 2026
侧面证明了一件事：整个行业都开始相信稀疏注意力是 dense attention 的解药。
SubQ 走的是「重训一个模型」，属于垂直整合，风险大回报也大。@evermind 的 MSA 走的是「给主流模型加记忆」，属于 水平嵌入，谁的模型都能用。
另外，SubQ API 跟 SubQ… https://t.co/yZ8IugqquAYeah, that's pretty cool—using $29 million to build a 12M context window, which indirectly proves one thing: the entire industry is starting to believe that sparse attention is the antidote to dense attention. SubQ takes the "retrain a single model" approach, which is vertical
-
GPT-5.5 Achieves Record Financial Document Extraction at Ramp
By
–
Next: @willkoh_kc from @tryramp tested GPT-5.5 inside their own harness.
— Romain Huet (@romainhuet) 5 mai 2026
“It was discovering ways to use the tools that we had given it… and figuring out novel ways to solve problems.”
He also tested it on financial-doc evals: GPT-5.5 hit their highest perfect extraction rate. pic.twitter.com/4C1N8QIrscNext: @willkoh_kc from @tryramp tested GPT-5.5 inside their own harness. “It was discovering ways to use the tools that we had given it… and figuring out novel ways to solve problems.” He also tested it on financial-doc evals: GPT-5.5 hit their highest perfect extraction rate.
-
HuggingFace bf16 Positional Embeddings Bug Went Unnoticed
By
–
Remember how HF precalculated positional embeddings in bf16 for a long time and the impact was small enough that it slipped through unnoticed? :O
-
Positional Embeddings Were Less Important Than Originally Thought
By
–
It actually turned out the particular spectrum of the chosen waves often is pretty horrible, but it also turned out positional embeddings weren't actually that important anyway so no-one really noticed for years.
-
Ilya’s Observation: Sam’s Behavior, Not AGI, Stood the Test of Time
By
–
Colonoscopy, cancer prevention, and the new arithmetic of benefit – @TheLancet “The study's 13-year results compel a recalibration of what colonoscopy can—and cannot—achieve at the population level.
-
GPT 5.5 Instant Achieves High Benchmark Performance, Reflecting AI Progress
By
–
All benchmarks are flawed, but GPQA has been fairly consistent & highly correlated with other measured benchmars. I think it's a good way to see how far we've come that the free model from OpenAI, GPT 5.5 Instant, is at a level that even paid models did not reach until late 2025
-
AI Reviewer Performance Comparison: Codex GPT-5.4 Tops Rankings
By
–
Now, now. Be nice. Greg has a been great witness!. For Elon.
-
MIT Plant-Inspired Robot Handles Objects with Care
By
–
This Plant-Inspired #Robot Handles Objects with Care
— Ronald van Loon (@Ronald_vanLoon) 5 mai 2026
by @MIT#Robotics #Engineering #ArtificialIntelligence #Innovation #Technology pic.twitter.com/IpDoXtRMOaThis Plant-Inspired #Robot Handles Objects with Care
by @MIT #Robotics #Engineering #ArtificialIntelligence #Innovation #Technology
