biggest questions in ai: – since the transformer, most AI improvement was via scaling up & great engineering
– almost no algorithmic ideas have driven performance, besides RLHF
– future gains will likely require algorithmic innovation how long will they take? who does them?

→ View original post on X — @alexandr_wang,