In February 1994, Martijn Koster sold early web developers on robots.txt with one line: it "doesn't require any server/client/protocol changes." A site owner posted a text file naming pages that automated crawlers should skip. Nothing else had to change.
Kevin Kelley's October 30 Internet-Draft makes the same offer for AI training. Robots.txt groups its rules under crawler names. Kelley invents a name no crawler carries, __usage_ai-training, and puts ordinary Allow and Disallow lines under it. Older software sees instructions for some other crawler and skips them.
As in 1994, cost decides where the rule goes. A preference about what models may learn from now lives in a file built to say which pages may be downloaded. It counts only where software learns the new name. Google says it aims to start in 2027.
Status: Revision -00, submitted by one person. No IETF working group has adopted it. It expires May 3, 2027.
Google: "In 2027," per its announcement. Its crawler documentation says early 2027. Both describe a plan. Neither announces a launch.
Example: Under the new name, Disallow: / plus Allow: /press-kit/ permits training on the press kit only.
Wildcard excluded: A User-Agent: * group says nothing about training. The site must write out the new name.
No group, no preference: If a site has no group under the new name, it has said neither yes nor no.
Crawling unaffected: The group governs how content is used. Crawl rules elsewhere in the file still decide which pages get fetched.
Google-Extended: If a site hasn't set Google's own AI opt-out, Google will count a refusal under the new name as one. Search ranking is unaffected.
