Have you tried seeing what impact different amounts of extra tokens has? Have you looked at the impact on downstream tasks too?
Token Budget Impact on Model Performance and Downstream Tasks
By
–
By
–
Have you tried seeing what impact different amounts of extra tokens has? Have you looked at the impact on downstream tasks too?