Dispelling a popular myth about letter-counting issues in LLMs by asking ChatGPT: h·o·w m·a·n·y R’s a·r·e i·n “s·t·r·a·w·b·e·r·r·y”?
@goodside
-
LLMs freestyle rap answers without pause or revision
By
–
autoregressive LLMs spend the same amount of compute to produce each token, emitting steadily where a human would pause for calculation, thought. they cannot backspace or revise. LLMs don’t write their answers; they freestyle rap them on livestream
-
Asking o1-preview how many r’s are in strawberry
By
–
Asking ChatGPT o1-preview
¿,ʎɹɹǝqʍɐɹʇs, uᴉ ǝɹɐ s,ɹ ʎuɐɯ ʍoɥ -
LLMs struggle with word length and final letters
By
–
It’s not always right of course — the other three answers I saw in five samples were “facetiously” (twice) and “abstentious.” But 2/5 here is pretty good — LLMs are famously bad at identifying the length or final letters of words
-
Prompting ChatGPT-4o with chain-of-thought reduces correctness
By
–
For comparison, when I give the same prompt to ChatGPT 4o, but replacing the last sentence with “Think step-by-step,” it returns 0 correct answers in 10 samples: “salubrious,” “abstemious” (x4), “illegible,” “repetitive,” “luminously”, “abstinential,” and “inexpensive.”
-
Commentary on ChatGPT, Claude-style LLMs and o1
By
–
Appreciate the kind words but I also wasn’t wild about ChatGPT at launch so my takes may be a Jim Cramer anti-signal I’m also in awe of o1 as much as anyone, but you can’t quite treat it as a refinement to 4o/Claude-style LLMs. It’s something new, experimental — still raw
-
Debate over letting LLMs ‘think’ and hallucinate
By
–
everyone: You’re stifling the raw, beautiful creativity of LLMs! They need space to think, to *hallucinate* — to not live in constant fear of our pink-slime opinions! OpenAI: ok we gave it a spot to think without worrying about you watching everyone: nooo u monsters how could u
-
Personal take on prompt engineering and prompting
By
–
o1 prompting is alien to me. Its thinking, gloriously effective at times, is also dreamlike and unamenable to advice. Just say what you want and pray. Any notes on “how” will be followed with the diligence of a brilliant intern on ketamine.
-
Three Types of Effective LLM Prompts
By
–
Every LLM prompt is at most two: – A realistic, economically valuable task
– A task differentiating frontier models
– Something you’d gladly read in a tweet -
ChatGPT notes LLM limits for counting and spelling
By
–
ChatGPT o1-preview explains that LLMs cannot reliably count or spell, and are not appropriate tools for such tasks, in a poem where the first letter of each word spells: "Scalable inference is all you need; Strawberry fields forever."