The string “” is handled specially by GPT-3’s tokenizer to represent the end of a generation. Whenever this token is emitted, the model stops talking immediately:
LLMS
-
ChatGPT silently removes empty quote substrings from input
By
–
A demonstration that ChatGPT silently removes from user input all substrings of form “” where “foobar” is any single word without whitespace:
-
Tokenization issue in counting letters of word ‘elementary’
By
–
Yes. The correct response to: How many letters are in “elementary”? …really should be: Due to tokenization issues you can read about [here], I need to count these letters one at a time: e1-l2-e3-m4-e5-n6-t7-a8-r9-y10. Thus we see ‘elementary’ has 10 letters.
-
Simulating Reality: LLMs and the Simulation Hypothesis Question
By
–
The reason that emulating virtual machines, chatrooms, and APIs inside large language models is exciting, isn’t just because “haha cool nerd stuff”. It means we are that much closer to simulating all of our world in it. And if we can do it… how would we know we aren’t in one?
-
Whisper Fine-tuning Event with OpenAI and Meta Talks
By
–
It's the 5th & the Whisper fine-tuning event (powered by @LambdaAPI
) is officially a go!! As part of the launch, we'll be hosting talks today at 17:30 CET 1. @OpenAI
's @_jongwook_kim on the Whisper architecture.
2. @MetaAI
's @ChanghanWang on the VoxPopuli dataset. -
Optimizing AI Chat Cost Per Conversation Economics
By
–
average is probably single-digits cents per chat; trying to figure out more precisely and also how we can optimize it
-
Safety Guidelines Modified: Known AI Jailbreaks No Longer Work
By
–
not retraining doesnt mean not modifying the safety guidelines, which is what people have been observing (see the links, a lot of the known jailbreaks dont work anymore)
-
Remarkable harmlessness of ChatGPT overshadowed by shifted goalposts
By
–
The posts of "Look, I got ChatGPT to be evil!" are missing the point. It's a remarkable achievement that it seems to never reply to an innocent question with a harmful answer. It's sad the goal posts shifted so fast that nobody cared when it happened.
-
Initial shock at LLM code errors fades with experience
By
–
How much of that is just novelty, though? The first time I noticed an error in LLM code I trusted, it felt fundamentally unfair — as though the code were maliciously forged rather than just wrong. But you develop an intuition for it eventually.