Yeah that makes sense to me – I've been doing some LLM-as-judge thing decently and having it provide the rationale before the winner does appear to provide better results
@simonw
-
Non-reasoning AI models excel at step-by-step problem solving
By
–
My experience is that these days most good non-reasoning models will spot if a problem can benefit from thinking step by step and just do it without you telling them to
-
Tailoring LLM Prompts to Your Knowledge Level Strategy
By
–
I do the reverse of that sometimes – "I'm an experienced Python programmer with limited knowledge of TypeScript" to get answers that are better tailored to my own knowledge
-
GraphQL N+1 Query Problems: Backend Performance Challenges
By
–
It's a recipe for N+1 query problems – implementing a performant GraphQL backend is actually really hard, especially when the frontend team can arbitrarily change the query patterns any time they want to
-
Tell LLMs Exactly What You Want for Better Code Generation
By
–
Yeah, for coding I usually tell the model exactly what I want it to write https://
simonwillison.net/2025/Mar/11/us
ing-llms-for-code/#tell-them-exactly-what-to-do
… -
Minimal Prompting: The Art of Simplicity in AI Prompts
By
–
I'm a big fan of what I call "minimal prompting": try to find the shortest, simplest prompt that can achieve your goal and then iterate on it and expand it only if you don't get the desired result from the simple version
-
Prompt Hacks Effectiveness: Model Evolution and Measurement Challenges
By
–
The problem with prompting hacks like "I'll tip you $200" is that they mostly emerged a year or two ago and models have evolved a lot since then It's very easy to remain superstitious about them though, especially as measuring their effectiveness is actually very difficult
-
Comparative prompt execution tracking for benchmark evolution
By
–
I'd love to be able to buy a one-off comparative prompt execution in a few months just to see how a new benchmark has evolved over time
-
Dynamic Auction Queue System for Claude Sonnet API Access
By
–
Here's a wild idea for you: keep just one server running Claude 3 Sonnet online, hook up a job queue with a limited length and have an auction mechanism people can use to buy their place in the queue such that price varies with demand
-
xAI Deep Search vs Claude Research: AI Giants Launch Competing Tools
By
–
xAI called their equivalent Deep Search (launched with Grok 3 on Feb 19th) Anthropic called theirs "Claude Research" (April 15th)