Build better base models. “there exists a substantial capabilities gap between open and closed LMs that, with current methods, can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs.”

→ View original post on X — @nathanbenaich