Loading
AI crawler comparison
Blocking one does not block the other — training and ChatGPT search are separate opt-outs.
Short answer
They are not interchangeable. GPTBot handles AI model training, OAI-SearchBot handles AI search and live answers. Blocking one has no effect on the other, so a robots.txt rule aimed at the wrong agent produces exactly the outcome you were trying to avoid.
| GPTBot | OAI-SearchBot | |
|---|---|---|
| Operator | OpenAI | OpenAI |
| Purpose | AI model training | AI search & live answers |
| robots.txt | Respects robots.txt | Respects robots.txt |
| Published IP ranges | Yes — verifiable | Yes — verifiable |
| Official docs | Yes | Yes |
| Blocking costs you | No traffic — training only | AI-answer citations |
GPTBot is OpenAI's web crawler for gathering training data. Content it collects may be used to improve future GPT models. It is OpenAI's broadest crawler, and the one most site owners mean when they talk about "blocking ChatGPT" — though blocking it does not remove you from ChatGPT's live search answers, which use OAI-SearchBot instead.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: GPTBot.
OAI-SearchBot builds the index behind ChatGPT's search experience — the linked, cited answers ChatGPT gives for current-events and lookup queries. Blocking it removes your pages from those citations without affecting model training, which is controlled separately by GPTBot.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: OAI-SearchBot.
User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Disallow: /
The split most sites want when the two agents do different jobs: opt out of one without giving up the other.
User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Allow: /
GPTBot is OpenAI's AI model training agent — Crawls content to train OpenAI's models. OAI-SearchBot is OpenAI's AI search and live answers agent — Powers search-style results surfaced in ChatGPT. They are separate user agents with separate robots.txt rules.
No. robots.txt matches on the user-agent token, so a group naming GPTBot applies only to GPTBot. OAI-SearchBot reads the group that names it, or the wildcard group if none does. To stop both you need a rule for each — or a wildcard rule, which would also affect every other crawler that reads it.
It depends which outcome you want. Blocking GPTBot opts you out of OpenAI's model training and costs you no traffic today, because training crawlers send no visitors. Blocking OAI-SearchBot removes you from the answers OpenAI's assistant shows its users. The common choice is to block training agents and allow search agents, so your content stays citable without feeding model training.
GPTBot: OpenAI documents GPTBot's IP ranges and states it honors robots.txt disallow rules. OAI-SearchBot: OpenAI states OAI-SearchBot respects robots.txt rules.
Yes — they send different user-agent strings, so grep for each token separately: grep -i "GPTBot" and grep -i "OAI-SearchBot". Be aware that a user-agent header is self-declared and anything can send either string. For a check that cannot be spoofed, match the request IP against the published ranges: GPTBot at https://openai.com/gptbot.json, OAI-SearchBot at https://openai.com/searchbot.json.
Part of our directory of every known AI crawler. Last verified: 2026-09-13.