and as you may have already guessed, being the best small-sized encoder means NeoBERT is a great embedding model. it's state-of-the-art on MTEB for its parameter class (more on this later!):
RESEARCH
-

NeoBERT: Iterative Deep Learning Breakthroughs Through Experimentation
By
–
like many important advancements in deep learning, NeoBERT arose from running lots of tiny experiments, learning from them, and stacking the results together into something that works really well:
-

NeoBERT: New State-of-the-Art Small Encoder Model
By
–
In the beginning, there was BERT. Eventually BERT gave rise to RoBERTa. Then, DeBERTa. Later, ModernBERT. And now, NeoBERT. The new state-of-the-art small-sized encoder:
-

Optimal Depth-Width Ratio Improves Transformer Performance
By
–
the key insight, i think, is using an optimal depth-to-width ratio for the transformer architecture. and training on good data. a lot of good data. even though NeoBERT has slightly more parameters, it's still faster AND more effective than ModernBERT for long sequences:
-

The GrandMaster Has Arrived: AGI Breakthrough Announcement
By
–
GM The GrandMaster Has Arrived. #AGIALPHA #ASIFirst
-
LLMs now understand physics, disproving skeptics
By
–
One year ago, LLM skeptics were saying "image models do not understand physics and never will", and they would cite as examples messed up images of gymnasts.
— m_ric (@AymericRoucher) 24 juin 2025
Now it appears LLMs can represent complex dynamics accurately. So they must 'understand' physics to some extent.
I think… https://t.co/tSxBkLxwRpOne year ago, LLM skeptics were saying "image models do not understand physics and never will", and they would cite as examples messed up images of gymnasts. Now it appears LLMs can represent complex dynamics accurately. So they must 'understand' physics to some extent. I think
-
AI Intelligence Surge: Preparing for Exponential Advancement
By
–
L’IA sera bientôt 1000 fois plus intelligente que les plus intelligents d’entre nous ! Beaucoup sont en plein déni et se réfugient derrière les paroles rassurantes de gens qui n’ont jamais touché ChatGPT… et minimisent son impact Il faut vous préparer ! Il y a le feu au lac
-
Europe’s Digital Decade 2025: Shaping the Future Strategy
By
–
Digital Decade 2025 – Special Eurobarometer | Shaping Europe’s digital future https://
digital-strategy.ec.europa.eu/en/library/dig
ital-decade-2025-special-eurobarometer
…
#digital #europe #digitaldecade #innovation #technology @DigitalEU @ArturHabant @elaniazito @AkwyZ @antgrasso @AnthonyRochand @CurieuxExplorer @Shi4Tech @BetaMoroney @enilev -

International cooperation shapes inclusive equitable development-oriented AI
By
–
As a precursor to the India- AI Impact Summit 2026, this high-level session will bring together global voices, including ministers, senior diplomats, officials, and ambassadors, to discuss how international cooperation can shape inclusive, equitable, and development-oriented AI
-

RASP Programming Language Compiles Algorithms to Transformer Weights
By
–
another incredibly underrated paper: Thinking Like Transformers (Weiss et al, 2021) presents RASP: a programming language that compiles to transformer *weights*. can implement sort(), bincount(), etc. seems important. why don't interpretability people care about this?
