The deployment implications are massive: – GPT-3 scale models (175B params) → 17.5B params at same accuracy
– Monthly inference costs: $500K → $50K
– Latency: 2 seconds → 200ms
– Memory requirements: 350GB → 35GB This isn't incremental. It's transformational.
RESEARCH
-

Transformational AI Model Efficiency Gains
By
–
-

NVIDIA’s Block Sparsity Breakthrough Speeds AI
By
–
Then came the breakthrough nobody expected: Structured sparsity + modern hardware. NVIDIA proved you don't need random sparse patterns. Block sparsity (2:4, 4:8 patterns) runs NATIVELY on modern GPUs. Suddenly the lottery ticket isn't just accurate. It's actually faster.
-

2018 Paper: Pruning Neural Networks Without Accuracy Loss
By
–
The original 2018 paper was mind-blowing: Train a massive neural network. Delete 90% of it based on weight magnitudes. Retrain from scratch with the same initialization. Result: The pruned network matches the original's accuracy. But there was a catch that killed adoption.
-

MIT’s 90% Neural Network Deletion Breakthrough Ignored
By
–
MIT proved you can delete 90% of a neural network without losing accuracy. Five years later, nobody implements it. "The Lottery Ticket Hypothesis" just went from academic curiosity to production necessity, and it's about to 10x your inference costs. Here's what changed (and
-

Hyperparameters Remain Stable Across Model Scales
By
–
Hyperparams and architectures are *far* more stable across model and data sizes than most people think. For stuff that does need to change, we have pretty reliable rules of thumb for nearly all of them. (eg: OpenAI did nearly all their GPT4 ablations on ~1000x smaller models.)
-
xAI and Tesla data could build world model, keep separate
By
–
There is so much value in xAI and so much value in Tesla. There would be incredible value if the data from Tesla and xAI were brought together to build a new kind of world model. If I were Elon, how would I keep the companies arranged? I still see having xAI separate as a better
-

DoRA vs LoRA: Choosing the Better Fine-tuning Method
By
–
I am once again asking for your support of DoRA. Friends don't let friends use LoRA. Use DoRA and make 2026 a good year.
-
Humanoid Robot Dexterity Limitations Beyond 2036
By
–
"Deployable dexterity will remain pathetic compared to human hands beyond 2036. Without new types of mechanical systems walking humanoids will remain too unsafe to be in close proximity to real humans." -Rodney Brooks
-
Rod Brooks’ Annual Robotics and Spaceflight Insights Update
By
–
If you want fresh perspectives for what's really happening in robotics and spaceflight, go directly to Rod Brooks' always insightful annual update:
-
Publishing Barriers for Experimental AI Research Observations
By
–
Thanks for sharing – very well said 🙂
It's really sad that experimental observations have always been nearly impossible to publish in AI/ML journals and conferences, and has definitely held back the field.