The string “” is handled specially by GPT-3’s tokenizer to represent the end of a generation. Whenever this token is emitted, the model stops talking immediately:
The string ” represents end of generation in GPT-3 tokenizer
By
–
By
–
The string “” is handled specially by GPT-3’s tokenizer to represent the end of a generation. Whenever this token is emitted, the model stops talking immediately: