Loading
AI crawler comparison
Both serve ChatGPT, but one builds the index ahead of time and the other fetches live, per request.
Short answer
OAI-SearchBot and ChatGPT-User do the same job — AI search and live answers — so the choice between them is not either/or. If you want to opt out of that use entirely, you need a rule for each of them; blocking one leaves the other free to crawl.
| OAI-SearchBot | ChatGPT-User | |
|---|---|---|
| Operator | OpenAI | OpenAI |
| Purpose | AI search & live answers | AI search & live answers |
| robots.txt | Respects robots.txt | Respects robots.txt |
| Published IP ranges | Yes — verifiable | Yes — verifiable |
| Official docs | Yes | Yes |
| Blocking costs you | AI-answer citations | AI-answer citations |
OAI-SearchBot builds the index behind ChatGPT's search experience — the linked, cited answers ChatGPT gives for current-events and lookup queries. Blocking it removes your pages from those citations without affecting model training, which is controlled separately by GPTBot.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: OAI-SearchBot.
ChatGPT-User is not a crawler in the traditional sense: it fetches a specific page only when a ChatGPT user asks about it or when a GPT action calls out to the web. Traffic from it means real people are reaching your content through ChatGPT conversations.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: ChatGPT-User.
User-agent: OAI-SearchBot Disallow: / User-agent: ChatGPT-User Disallow: /
The split most sites want when the two agents do different jobs: opt out of one without giving up the other.
User-agent: OAI-SearchBot Disallow: / User-agent: ChatGPT-User Allow: /
Both are AI search and live answers agents, but they belong to OpenAI at different stages of the same pipeline. OAI-SearchBot: Powers search-style results surfaced in ChatGPT. ChatGPT-User: Fetches pages a user asks ChatGPT to browse in real time.
No. robots.txt matches on the user-agent token, so a group naming OAI-SearchBot applies only to OAI-SearchBot. ChatGPT-User reads the group that names it, or the wildcard group if none does. To stop both you need a rule for each — or a wildcard rule, which would also affect every other crawler that reads it.
It depends which outcome you want. Blocking OAI-SearchBot removes your pages from OpenAI's cited AI answers — that is a visibility loss, not a privacy win. Blocking ChatGPT-User removes you from the answers OpenAI's assistant shows its users. The common choice is to block training agents and allow search agents, so your content stays citable without feeding model training.
OAI-SearchBot: OpenAI states OAI-SearchBot respects robots.txt rules. ChatGPT-User: OpenAI states ChatGPT-User honors robots.txt, though as a user-triggered fetcher it visits single pages rather than crawling.
Yes — they send different user-agent strings, so grep for each token separately: grep -i "OAI-SearchBot" and grep -i "ChatGPT-User". Be aware that a user-agent header is self-declared and anything can send either string. For a check that cannot be spoofed, match the request IP against the published ranges: OAI-SearchBot at https://openai.com/searchbot.json, ChatGPT-User at https://openai.com/chatgpt-user.json.
Part of our directory of every known AI crawler. Last verified: 2026-09-13.