That's interesting – I haven't seen that done before. So did you just add a few tokens to a pretrained model, and then fine-tune them somehow? Or left them as randomly initialised?

→ View original post on X — @jeremyphoward