I’m not convinced many-shot examples degrade performance. Artificial distractor text will, as will long inputs, but many (diverse, reasonable) examples in traditional k-shot ICL should strictly help performance at the (now much lower) expense of latency/cost.

→ View original post on X — @goodside