Does it actually work in practice? Wikitext-2 (language modeling): 13M param Grassmann model: 275.7 perplexity
13M param Transformer: 248.4 perplexity
Gap: ~11% behind SNLI (natural language inference): Grassmann head: 85.50% validation accuracy
Transformer head: 85.45%
RESEARCH
-

Grassmann model trails in perplexity but leads in SNLI accuracy
By
–
-

Attention quadratic vs Grassmann linear scaling complexity gap
By
–
The complexity story is WILD: Standard attention: O(L²d) – quadratic scaling with sequence length
Grassmann mixing: O(Ld²) – LINEAR scaling (for fixed rank r) This isn't just theoretical. As sequences get longer, the gap becomes exponential. -

Grassmann flows replace attention with geometric evolution
By
–
The alternative is mathematically beautiful: Grassmann flows replace attention with controlled geometric evolution on a manifold. – Reduce hidden states to r=32 dimensions (from d=256)
– Encode token pairs as 2D subspaces on Gr(2,r)
– Use Plücker coordinates to create geometric -

Massive attention matrices cause Transformer interpretability crisis
By
–
Here's the interpretability crisis no one talks about: Every Transformer layer creates an L×L attention matrix. Across 96 layers and 96 heads in GPT-4, that's a MASSIVE evolving tensor cloud. The paper proves this is why we can't interpret these models not their size, but their
-

Transformer performance matched without computing attention weights
By
–
This paper just murdered the foundation of every AI model you've ever used. A researcher proved you can match Transformer performance WITHOUT computing a single attention weight. Here's what changed (and why this matters now):
-

Cisco AI Summit: Enterprise AI Building, Security, and Scaling
By
–
Join us online for the Cisco AI Summit livestream. If you care about how enterprise AI is actually being built, secured, and scaled, this is a day worth putting in the diary. Cisco is bringing together many of the people shaping the next wave of AI, from frontier models and the
-
K2.5 Agent Swarm Creates New Language for Deep Sea Species
By
–
K2.5 Agent Swarm designed an entirely new language in just 38 minutes for an intelligent species that lives in the deep sea and communicates by emitting light through its skin.
— 机器之心 JIQIZHIXIN (@jiqizhixin) 27 janvier 2026
(The video is sped up 20×.) https://t.co/MNlgtB4fAD pic.twitter.com/76HtUFdGp5K2.5 Agent Swarm designed an entirely new language in just 38 minutes for an intelligent species that lives in the deep sea and communicates by emitting light through its skin. (The video is sped up 20×.)
-

Derf: Replacing Transformer Normalization with Simpler Function
By
–
What if you could replace a core part of a Transformer with something simpler and stronger? Researchers from Princeton, NYU, and CMU present Derf. They swapped the standard "normalization" layer with a simple, element-by-element function called Derf (based on a Gaussian error
-
State-of-the-art reinforcement learning algorithms replacing traditional methods
By
–
#PaperADay 11
Discovering state-of-the-art reinforcement learning algorithms https://nature.com/articles/s41586-025-09761-x
… The paper is about replacing the traditional RL target generation algorithms (policy gradient, GAE, TD-lambda, etc) that look at a sequential block of agent frame predictions -
Future 5.X versions aim to improve writing and general-purpose models
By
–
"We will make future versions of 5.X much better at writing" "The future is in general-purpose models"