Loading
AI crawler comparison
Perplexity's index crawler versus its live fetch on behalf of a searcher.
Short answer
PerplexityBot and Perplexity-User do the same job — AI search and live answers — so the choice between them is not either/or. If you want to opt out of that use entirely, you need a rule for each of them; blocking one leaves the other free to crawl.
| PerplexityBot | Perplexity-User | |
|---|---|---|
| Operator | Perplexity | Perplexity |
| Purpose | AI search & live answers | AI search & live answers |
| robots.txt | Partially respects robots.txt | Ignores robots.txt |
| Published IP ranges | Yes — verifiable | Yes — verifiable |
| Official docs | Yes | Yes |
| Blocking costs you | AI-answer citations | AI-answer citations |
PerplexityBot builds the index behind Perplexity's cited, search-style answers. Perplexity is an answer engine rather than a model trainer, so being crawled by PerplexityBot is primarily about visibility: it determines whether your pages can appear as sources in Perplexity results.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: PerplexityBot.
Perplexity-User fetches a specific page when a Perplexity user's question requires it. Because each fetch is directly user-initiated, Perplexity treats it like a browser rather than a crawler — which is also why it generally does not obey robots.txt.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: Perplexity-User.
User-agent: PerplexityBot Disallow: / User-agent: Perplexity-User Disallow: /
The split most sites want when the two agents do different jobs: opt out of one without giving up the other.
User-agent: PerplexityBot Disallow: / User-agent: Perplexity-User Allow: /
Both are AI search and live answers agents, but they belong to Perplexity at different stages of the same pipeline. PerplexityBot: Crawls and indexes pages for Perplexity's answer engine. Perplexity-User: Fetches a page a user asks Perplexity to browse live.
No. robots.txt matches on the user-agent token, so a group naming PerplexityBot applies only to PerplexityBot. Perplexity-User reads the group that names it, or the wildcard group if none does. To stop both you need a rule for each — or a wildcard rule, which would also affect every other crawler that reads it.
It depends which outcome you want. Blocking PerplexityBot removes your pages from Perplexity's cited AI answers — that is a visibility loss, not a privacy win. Blocking Perplexity-User removes you from the answers Perplexity's assistant shows its users. The common choice is to block training agents and allow search agents, so your content stays citable without feeding model training.
PerplexityBot: Perplexity says PerplexityBot respects robots.txt, but independent investigations (notably Cloudflare's 2024–2025 reports) have observed undeclared fetching that bypassed blocks. Perplexity-User: Perplexity's own documentation says Perplexity-User generally ignores robots.txt because fetches are user-requested.
Yes — they send different user-agent strings, so grep for each token separately: grep -i "PerplexityBot" and grep -i "Perplexity-User". Be aware that a user-agent header is self-declared and anything can send either string. For a check that cannot be spoofed, match the request IP against the published ranges: PerplexityBot at https://www.perplexity.ai/perplexitybot.json, Perplexity-User at https://www.perplexity.ai/perplexity-user.json.
Part of our directory of every known AI crawler. Last verified: 2026-09-13.