They have an option to block AI-specific crawlers but still allow search engine crawlers, which matters if you want people to find your content via search
@simonw
-
AI Tools Empower Developers to Scale Ambitious Projects
By
–
I don't think AI will replace developers, but I do think it's already changing the work we do – devs who know how to effectively use these tools can take on more ambitious projects and change their pace of delivery
-
OpenAI Codex CLI Tool Built on Deprecated Model
By
–
Oh no! It uses the "Codex CLI" unofficial name we've been assigning to the http://
GitHub.com/openai/codex tool AND it's called that because it was written against the old (and now long departed) Codex model -
GitHub Copilot brand confusion versus OpenAI Codex
By
–
And to be fair Codex has nothing on Microsoft/GitHub Copilot when it comes to massive brand confusion these days
-
OpenAI Codex: Complete Collection of References
By
–
Here's my full collection of things they've called Codex https://
simonwillison.net/2025/May/16/op
enai-codex/
… -
Codex Asynchronous Coding Agent Web Application Launch
By
–
… which is somewhat ironic given Sam tweeted this the day before they released the Codex asynchronous coding agent web application (the CLI tool had been out for a month by that point)
-
OpenAI Codex Naming Confusion: CLI Tool vs Async Agent
By
–
OpenAI's decision to give their Codex CLI tool and their Codex asynchronous coding agent the exact same name means that when people say "have you tried Codex?" i have genuinely no idea which one they are talking about
-
Claude System Prompts Evaluation Examples Detailed
By
–
I quote that tweet pretty often! I'm looking for the next level of detail from @AmandaAskell though… I want to see just one example of an actual eval used for the Claude system prompts
-
Claude 4 Efficient Token Processing in Code
By
–
I think Claude 4 in Claude Code should be smart enough to grep the log when it needs to and consume just a few hundred tokens at a time
-
Native Problem-Solving in AI Systems Reduces Operational Friction
By
–
Much less hassle if it knows how to solve this problem out of the box – I've run servers for it and logged them to files and told it to read the logs but it's a lot of extra friction