“At the time, @SakanaAILabs only had 16 GPUs, which cost $30K per month. That’s pennies compared to the $100M+ it took to train advanced models like GPT-4… Model merging isn’t perfect though. @SakanaAILabs is one of only a few firms who have attempted to automate this process.”

→ View original post on X — @hardmaru,