Here's how to start exploring the vocabulary of the tokenizer – interesting to see they have tokens like [TOOL_CALLS], [AVAILABLE_TOOLS], [/AVAILABLE_TOOLS], [TOOL_RESULTS] and [/TOOL_RESULTS] in there
Global AI News Aggregator
By
–
Here's how to start exploring the vocabulary of the tokenizer – interesting to see they have tokens like [TOOL_CALLS], [AVAILABLE_TOOLS], [/AVAILABLE_TOOLS], [TOOL_RESULTS] and [/TOOL_RESULTS] in there
Leave a Reply