4). Mixtures of In-Context Learners – uses subsets of demonstrations to train experts via in-context learning; given a training set, a trainable weighting function is used to combine the experts' next-token predictions…

→ View original post on X — @dair_ai