Intelligence density per unit of memory & compute is a fundamental metric. My rough guess is that large models are at least one, but maybe two, orders of magnitude away from the Platonic maximum.

→ View original post on X — @elonmusk,