ClaudeBot is Anthropic's training crawler.
Collects web content that could contribute to training Anthropic's models.
Source: Anthropic: Does Anthropic crawl data from the web?- Vendor
- Anthropic
- Type
- Training. Collects pages to train AI models.
- User agent contains
- ClaudeBot
- robots.txt token
- ClaudeBot
- Published IP ranges
Note from our list: claude.com/crawling/bots.json is one shared list for all Anthropic bots.
Blocking ClaudeBot tells Anthropic to leave the site's future content out of its training data. Anthropic says its bots honour robots.txt and also support the non-standard Crawl-delay directive.
To block ClaudeBot:
User-agent: ClaudeBot Disallow: /
To allow it explicitly:
User-agent: ClaudeBot Allow: /
Is that request really ClaudeBot?
- Take the client IP from your server or CDN log, not from a header the client can set.
- Check it against Anthropic's published range listed above, or paste it into the AI bot IP verifier.
- A ClaudeBot user agent from an address that fails these checks is someone else using the name.
Other Anthropic bots
From Renown's bot list, updated 10 November 2026. See all 169
AI crawler checker
See what AI bots get from your site
Enter a URL and see, bot by bot, whether robots.txt lets it in, whether a firewall turns it away, and how much of the page it can read without running JavaScript.
AI visibility
Find out whether AI recommends you
Being readable is only the start. The free AI visibility check asks AI the questions your customers ask, shows the answers word for word, and tells you whether you were named.
Questions people ask
What is ClaudeBot?
ClaudeBot is Anthropic's training crawler. Collects web content that could contribute to training Anthropic's models.
Does ClaudeBot follow robots.txt?
Blocking ClaudeBot tells Anthropic to leave the site's future content out of its training data. Anthropic says its bots honour robots.txt and also support the non-standard Crawl-delay directive.
How do I check a ClaudeBot request is real?
Check the request's IP against Anthropic's published ranges.