Can AI read your site? Check before you guess.
An AI assistant can't quote a page its bot never fetched. A robots.txt line or a firewall rule turns those bots away, a page that needs JavaScript shows them very little, and nobody gets told. These tools show you what happens. They're free and you don't need to give us an email.
- AI crawler checkerEnter a site. We read its robots.txt the way GPTBot, ClaudeBot, PerplexityBot and the rest would, fetch the homepage as a browser and as four AI bots, and tell you which ones get turned away and what they can read without JavaScript.
- AI bot IP verifierPaste an IP from your logs. We check it against the ranges OpenAI, Anthropic, Perplexity, Google and others publish, so you can tell a real GPTBot from a scraper using its name.
- robots.txt generator for AI botsPick what to allow: training, search, or bots that fetch a page when someone asks. Get a block to paste into robots.txt, and a plain account of what each choice costs you.
- AI crawler directory169 AI crawlers and fetchers: who runs each one, what it's for, the user-agent token and how to verify it.
GPTBot and ChatGPT-User are not the same bot
OpenAI runs three. GPTBot collects pages that may be used to train its models. OAI-SearchBot builds the index behind ChatGPT search. ChatGPT-User turns up when a person in ChatGPT does something that needs your page. OpenAI says each robots.txt setting is independent, so blocking GPTBot keeps you out of training without taking you out of ChatGPT search.
Anthropic splits ClaudeBot, Claude-SearchBot and Claude-User the same way, and Perplexity, Google, Amazon, Meta and Mistral all run bots for different jobs. The tools here sort every bot into those three groups: training, search and user-triggered.
AI crawler directory
Look up a bot from your logs
Who runs it, what it collects, the user-agent token to use in robots.txt, and how to tell a real visit from one faking its user agent.
AI visibility
Find out whether AI recommends you
Being readable is only the start. The free AI visibility check asks AI the questions your customers ask, shows the answers word for word, and tells you whether you were named.