It's been a couple of weeks since my last newsletter (which is more-or-less copied and pasted from my blog), turns out I've been writing longer bookmark descriptions recently so the newsletter ended up huge, even though I didn't think I had much content https://
simonw.substack.com/p/llm-slop-dat
asette-secrets-llm-evals
…
@simonw
-
LLM Slop Datasets and Evaluation Secrets Newsletter
By
–
-
Static typing in interactive REPL environments
By
–
Interactive REPL tools like Jupyter or browser DevTools consoles are nicer to use without having to think about types Not sure I've ever tried a statically typed language in a REPL to be honest, do you know any good ones?
-
Deterministic Quoting: Preventing LLM Text Alterations
By
–
I saw an interesting proposal today for avoiding those kinds of minor alterations to direct quotes: https://
mattyyeung.github.io/deterministic-
quoting
… "That’s the only way to guarantee that an LLM has not transformed text: don’t send it through the LLM in the first place." -
Switching AI Models: Opus vs ChatGPT Code Interpreter
By
–
I think you misunderstood why I switched models – it wasn't because I got a bad result out of Opus, it was because I knew ChatGPT Code Interpreter could tackle the problem using tools that Opus doesn't have access to
-
OpenAI’s GPT2-Chatbot Revealed Through Rate Limit Error
By
–
Here's confirmation that gpt2-chatbot (now renamed to "im-also-a-good-gpt-chatbot") is from OpenAI, thanks to a revealing 429 rate limit error message
-
Understanding LLM Flaws: A Practical Guide to Tool Usage
By
–
The key with all of this is to treat LLMs as flawed tools If a tool is flawed it doesn't mean it's completely useless, it means you have to understand those flaws and how to work around them LLMs have almost no documentation about how to do that, which is why I write about them
-
Writing SQLite C Extensions on Mobile Devices
By
–
I certainly never would have dreamed of trying to write a C extension for SQLite before, and now it's the kind of thing I can prototype on my phone while out walking my dog
-
LLM Assistance Lowers Entry Barriers for Ambitious Projects
By
–
See also this post, where I talk about how LLM-assistance has lowered the barrier to entry for me on all sorts of other things (like using AppleScript) such that I now take on more ambitious projects
-
AI Tools Enable Spontaneous Geospatial Data Conversion Tasks
By
–
Did you see the bit about this being a side quest? If converting shape files to GeoJSON was an important task for the day then I might have approached this differently, but I now have tools that let me do this kind of thing on the spur of the moment, just for fun!
-
Datasette-secrets updates and mysterious video project announcement
By
–
Weeknotes: more datasette-secrets, plus a mystery video project that I'm not talking about yet