I'll believe they've done that when one of the models draws me a good pelican!
@simonw
-
Benchmark Joke Proves Surprisingly Effective for Model Evaluation
By
–
For a benchmark that was originally intended to be a joke I found it's surprisingly effective at quickly evaluating how good a model is!
-
Apple Silicon GPU Memory Access Enables Better LLM Performance
By
–
I don't think that will run an LLM very well, you need GPU-accessible memory to get good performance That's what's neat about Apple Silicon – regular RAM is available to the GPU
-
AI Labs Benchmark Goals: Pelican Bicycle Illustration Challenge
By
–
My secret goal here is to have AI labs waste copious amounts of time training for my benchmark… so I can finally get a decent illustration of a pelican riding a bicycle!
-
Qwen3-Coder-480B runs locally on high-end Mac Studio
By
–
Looks like Qwen3-Coder-480B-A35B-Instruct is a VERY strong coding model to run locally… if you can afford a ~$10,000 512GB Mac Studio to run it on!
-
Qwen Releases New Model While Notes Still Being Finalized
By
–
They released this new model literally as I was finishing typing up my notes on their new model from yesterday, Qwen3-235B-A22B-Instruct-2507 (I think yesterday's smaller model drew a better pelican)
-
Qwen releases major coding specialist model with strong benchmark results
By
–
Whoa. HUGE new coding specialist model from Qwen, accompanied by their own fork of gemini-cli (qwen-code) and some very impressive results on the various coding benchmarks It drew me an OK pelican
-
LLM Plugin Dependencies and Version Management
By
–
They are separate and you can usually update them separately, but sometimes a plugin may depend on a more recent version of LLM due to adding support for a new feature such as tools Check pyproject.toml in the plugin repo to see
-
Gemini-2.5-flash-lite now available in llm-gemini plugin
By
–
The new gemini-2.5-flash-lite model ID is now available in my LLM CLI tool / Python library via the updated llm-gemini plugin https://
github.com/simonw/llm-gem
ini/releases/tag/0.24
… -
Running Code Safely in Sandboxed Environments
By
–
Run code in a reliable sandbox on my own machine (with minimal setup required from me to get it working)