That's a good question! Well, at a minimum I'd say that what we're observing right now is that ICL is not as robust as we'd probably want it to be, as we don't always get the strictly monotonic improvement in performance that you'd expect from a normal learning algorithm.

→ View original post on X — @hugo_larochelle