“I would not trust any claims of a superior model until I see [both] ELO points on LMSys [and] private LLM evaluation from a trusted 3rd party, such as Scale AI's benchmark. The test set must be well-curated and held secret, otherwise it quickly loses potency.”
@goodside
-
Project Incorporates PlanSearch Ideas
By
–
Also see this project from @asankhaya
, which now incorporates the core ideas of PlanSearch (one day after the paper!) -
Softmax projection and difficulty of learning additions in embeddings
By
–
Each one still has the softmax issue though and it all gets turned into one vector at the end. Learning to do addition in that projection feels hard, but I don’t know enough to say it can’t happen
-
Question about softmax invariance and matching counts
By
–
I know there’s no integer count; my read was that softmaxing makes it invariant to the number of matches so it has to rely on other less reliable methods but I could well be misunderstanding that point; reading again now
-
When LLMs Should Use Code
By
–
Everyone acknowledges that code is the correct way to solve this. It’s still worth understanding why that’s necessary, and more practically for LLMs to understand *when* that’s necessary.
-
Paper on limitations of counting in single-token inference
By
–
Yes — and you can often achieve much more reliable counting by repeating an input a second time annotated with a running total, but that’s expensive and context-constrained; this paper is about limitations on counting directly in a single token inference
-
Why Transformers Struggle to Count, According to a Paper
By
–
The reason LLMs say there's two r's in "strawberry" isn't (just) tokenization — they struggle with counting generally, e.g. "horse" in the example shown. The paper in the quoted post below offers the best intuition I've seen for why this happens: Transformers can't count because
-
Claude may inject invisible copyright-avoidance instructions
By
–
Claude sometimes invisibly adds instructions to avoid reciting copyrighted works to the user’s prompt, presenting them to the model as though the user wrote them. Thus, these rules need not apply to Batman:
-
Commentary on LLM prompting and counting reliability
By
–
As many have pointed out, there are much better ways to prompt this. Also, it’s silly to use an LLM here, as there’s little hope of making them 100% reliable at counting. What’s notable IMO isn’t that it can’t count, but that it doesn’t see it can’t count (and e.g. try its REPL)
