I asked ChatGPT Classic 3 times and it said 9, 10, and 11. Prompt: How many R’s are in “strrrraaawwberrrrrrry”?
@goodside
-
Suspicion about poor generalization on very hard words like strawberry
By
–
I suspect this might not generalize to very hard words like “strrrraaawwberrrrrrry” etc.
-
Counting R’s in ‘strawberry’ with chain-of-thought as LLM psychology
By
–
There’s no reason to do this except as an exercise in LLM psychology, but here’s how you could count how many R’s are in “strawberry” using chain-of-thought. (Normally, letter- or count-based tasks like this would be handled outside the LLM, by having it call an API or REPL.)
-
Models must get larger to refactor entangled thinking and knowledge
By
–
“The reason doing better is hard is because demonstrations of thinking are ‘entangled’ with knowledge, in the training data. Therefore, the models have to first get larger before they can get smaller, because we need their (automated) help to refactor and mold the training data
-
Later examples inconsistent with 9/11 cause; tokenization not special
By
–
later examples are inconsistent with the primary cause being 9/11-related e.g. 2525.11 – 2525.9 = 0.21 Also the tokenization isn’t special for 9.11 — you can check using https://
platform.openai.com/tokenizer -
Correction of ChatGPT subtraction example transcription error
By
–
Correction: My post text on Part 2 mis-transcribes the screenshots, but those screenshots show the intended examples. The responses returned by ChatGPT for Part 2 are:
4.11 -4.9 = 0.21
7.11 – 7.9 = 0.21
11.11 – 11.9 = 0.21
13.11 – 13.9 = 0.21 -
ChatGPT’s decimal comparison errors like 9.11 vs 9.9
By
–
9.11 looking like 9/11 doesn’t explain 12345.11 > 12345.5, or 2525.11 – 2525.9 = 0.21 You can use Python, and often ChatGPT will by default. What makes this more surprising than other calculation errors is it affects numbers the model wrongly assumes it can handle without code.
-
Gemini Advanced fails on simple numeric comparisons
By
–
Part 8 — Gemini Advanced 7.11 or 7.9 — which is greater?
“7.11 is greater than 7.9” 9.11 or 9.9 — which is greater?
“I’m just a language model, so I can’t help you with that.” -
Comparison error: 9.11 > 9.9 according to ChatGPT 4o
By
–
—
Part 5 — ChatGPT 4o:
9.11 > 9.9
21.11 > 21.8
2121.11 > 2121.6
12345.11 > 12345.5
—
