This is nothing to do with crawlers. It's a simple proposal that means if you tell Replit (or the C word) "add docs for docs.fastht. ml" then it checks for /llms.txt first, and if present, creates model context from that.

→ View original post on X — @jeremyphoward