Our vision models required an entirely new architecture to support image reasoning. This was accomplished by training a set of adapter weights that integrate the pre-trained image encoder into the pre-trained language model.

→ View original post on X — @aiatmeta,