At small scale, at compute match, with enough supervised data points, a flan-t5 baseline is very tough to beat. Of course t5 has its problems (outdated etc) but a modern one would be very OP.

→ View original post on X — @yitayml,