AI Dynamics

Global AI News Aggregator

Managing Special Tokens in AI Models: Guidance Gaps

Has anyone seen published guidance about how to avoid tokens (including special things like [INST] or ) messing things up, for any of the available models? I've not seen good documentation on this myself yet

→ View original post on X — @simonw,

Commentaires

Leave a Reply

Your email address will not be published. Required fields are marked *