You can't merge the weights of different architectures with mergekit. I quite like this solution but I don't expect it to be competitive

→ View original post on X — @maximelabonne