AI crawler profile
What is Perplexity-User? How to allow or block Perplexity's crawler
Operator
Perplexity
Purpose
AI search & live answers
robots.txt
Ignores robots.txt
Official docs
Perplexity-User fetches a specific page when a Perplexity user's question requires it. Because each fetch is directly user-initiated, Perplexity treats it like a browser rather than a crawler — which is also why it generally does not obey robots.txt.
Does Perplexity-User respect robots.txt?
Perplexity's own documentation says Perplexity-User generally ignores robots.txt because fetches are user-requested.
Block Perplexity-User with robots.txt
User-agent: Perplexity-User Disallow: /
⚠ Because Perplexity-User's robots.txt compliance is unreliable, pair this with a firewall or CDN rule matching the user-agent string if blocking actually matters to you.
Explicitly allow Perplexity-User
User-agent: Perplexity-User Allow: /
Should you block Perplexity-User?
For most sites, no. Perplexity-User powers visibility: blocking it removes your pages from the cited answers Perplexity's assistant shows its users — the AI-era equivalent of de-indexing yourself from a search engine. Block it only if you deliberately don't want that audience. Our guide on blocking AI bots covers the trade-offs in detail.
Check and monitor Perplexity-User on your site
- AI Crawler Access Checker — see whether your current robots.txt allows or blocks Perplexity-User.
- robots.txt Generator for AI Bots — build a robots.txt with per-bot rules for all 15 known AI crawlers.
- AI Bot Log Analyzer — paste server logs and count real Perplexity-User hits, entirely in your browser.
Frequently asked questions
What is Perplexity-User?
Perplexity-User is Perplexity's agent for AI search and live answers. Fetches a page a user asks Perplexity to browse live.
Does Perplexity-User respect robots.txt?
Perplexity's own documentation says Perplexity-User generally ignores robots.txt because fetches are user-requested.
How do I block Perplexity-User?
Add "User-agent: Perplexity-User" followed by "Disallow: /" to your robots.txt file. Because this bot's robots.txt compliance is not reliable, enforce the block with firewall or CDN rules (for example a Cloudflare WAF rule matching the user agent) if it matters to you.
Does blocking Perplexity-User hurt my Google rankings?
No. Perplexity-User is separate from Googlebot, which handles Google Search indexing. Blocking Perplexity-User has no effect on your traditional search rankings, but it does remove your pages from the AI answers Perplexity's assistant serves to its users.
How can I tell if Perplexity-User is crawling my site?
Search your server access logs for the string "Perplexity-User". Our free AI Bot Log Analyzer does this in your browser: paste a log file and it counts hits per AI crawler, including Perplexity-User, with per-path breakdowns.
More Perplexity agents
Part of our directory of every known AI crawler, refreshed monthly. Last verified: 2026-07-15.