It's a good question. You're right, they're added in the background. In this particular example, I noticed that im_start wasn't properly added however. We still use "" as "bos_token".
BOS Token Configuration Issue in Language Models
By
–
By
–
It's a good question. You're right, they're added in the background. In this particular example, I noticed that im_start wasn't properly added however. We still use "" as "bos_token".