AI Dynamics

Global AI News Aggregator

About

Managing Special Tokens in AI Models: Guidance Gaps

Has anyone seen published guidance about how to avoid tokens (including special things like [INST] or ) messing things up, for any of the available models? I've not seen good documentation on this myself yet

→ View original post on X — @simonw