I am not sure "OpenAI ran with it" is entirely correct. I remember there was quite some rivalry between Google's encoder approach and OpenAI's push for decoder-style models.
Google tried to make encoder-style models work for many years, since the original architecture.
MACHINE LEARNING
-
Encoder vs Decoder: Google and OpenAI’s Architectural Rivalry
By
–
-
What Was Your Favorite AI Breakthrough This Year?
By
–
What was your favorite AI breakthrough this year?
-
DeepSeek V3.2 sparse attention mod overshadows throughput focus announcement
By
–
Based on the announcement post it seems the focus is token/sec throughput, cost, and latency. Not sure if that’s because the modeling performance wasn’t exceeding other models or if it was the original plan.
But even then there is now DeepSeek V3.2 with the sparse attention mod. -
Sebastian Raschka Updates Comprehensive LLM Architecture Comparison Article
By
–
Just updated the Big LLM Architecture Comparison article…
…it grew quite a bit since the initial version in July 2025, more than doubled! https://
magazine.sebastianraschka.com/p/the-big-llm-
architecture-comparison
… -
AI knows how I think and analyzes my X feeds
By
–
It knows how I think. Same with you. And it understands our content very deeply. So will be hyper amazing bringing you content. I am playing with AI to analyze my X feeds. See my post on that:
-
Architectural Variations in Modern Language Models
By
–
Yes, they are all relatively related, but usually they have a unique tweak like RMSNorm placement, sliding window, in this case MLA, etc.
I wrote about it in more detail here: -
Decades of Experience with Machine Learning Models Complexity
By
–
I know what you meant. But despite spending some decades doing that, I never got to the point that I found them lightweight, easy to understand, train, and infer. 🙂
-
Speed Challenge: Biotech Document Analysis Requires High-Performance Inference
By
–
8/
But there was a brutal roadblock: speed. Raycaster often needed to analyze thousands of hundred-page documents at once. If the system lagged, the entire experience fell apart.
Normal inference could not keep up with real biotech workloads. -
RL Environments Critical as Agents Move to Production
By
–
🤿 ICYMI: Agents don’t learn in the abstract — they learn by interacting with environments.
— Snorkel AI (@SnorkelAI) 12 décembre 2025
This post on RL environments is more relevant than ever as agents move into production and evaluation gets harder. pic.twitter.com/P96EfkYyBcICYMI: Agents don’t learn in the abstract — they learn by interacting with environments. This post on RL environments is more relevant than ever as agents move into production and evaluation gets harder.
-
Snorkeling in RL Environments: Training Agentic AI Systems
By
–
Worth a re-read https://
snorkel.ai/blog/snorkelin
g-in-rl-environments/
…
#AgenticAI #ReinforcementLearning #LLMs #AIResearch #AIInfrastructure