Researchers from Apple just uncovered scaling laws for native multimodal models. What did they find?
– No clear advantage for late-fusion over early-fusion.
– Early-fusion performs better at small scales.
– Add MoEs, and you unlock modality-specific weights→big performance gains

→ View original post on X — @jiqizhixin