What is new is just how many times this has been applied to LLaMA and other models to squeeze near-equivalent foundation model performance out of a small number of parameters. But that’s basically what happens when the cat comes out of the bag in open source.

→ View original post on X — @mattlynley