AI crawler profile
What is PanguBot? How to allow or block Huawei's crawler
Operator
Huawei
Purpose
AI model training
robots.txt
Compliance unknown
Official docs
None published
PanguBot is the user agent linked to data collection for Huawei's PanGu family of large language models. Unlike PetalBot (Huawei's documented search crawler), PanguBot has no official documentation, putting it in the same category as Bytespider: a training crawler you can only reliably control at the firewall level.
Does PanguBot respect robots.txt?
No official documentation exists and independent compliance reports are limited — pair any robots.txt block with a firewall or CDN rule if opting out matters to you.
Block PanguBot with robots.txt
User-agent: PanguBot Disallow: /
⚠ Because PanguBot's robots.txt compliance is unreliable, pair this with a firewall or CDN rule matching the user-agent string if blocking actually matters to you.
Explicitly allow PanguBot
User-agent: PanguBot Allow: /
Should you block PanguBot?
Blocking PanguBot opts your content out of Huawei's model training — a legitimate choice for original content you don't want reproduced by AI. The trade-off: models trained without your content are less likely to know your brand or recommend it unprompted. There is no direct traffic loss today, since training crawlers don't send visitors. See our guide on whether to block AI bots for the full decision framework.
Check and monitor PanguBot on your site
- AI Crawler Access Checker — see whether your current robots.txt allows or blocks PanguBot.
- robots.txt Generator for AI Bots — build a robots.txt with per-bot rules for all 28 known AI crawlers.
- AI Bot Log Analyzer — paste server logs and count real PanguBot hits, entirely in your browser.
Frequently asked questions
What is PanguBot?
PanguBot is Huawei's web crawler for collecting AI training data. Associated with training Huawei's PanGu large language models.
Does PanguBot respect robots.txt?
No official documentation exists and independent compliance reports are limited — pair any robots.txt block with a firewall or CDN rule if opting out matters to you.
How do I block PanguBot?
Add "User-agent: PanguBot" followed by "Disallow: /" to your robots.txt file. Because this bot's robots.txt compliance is not reliable, enforce the block with firewall or CDN rules (for example a Cloudflare WAF rule matching the user agent) if it matters to you.
Does blocking PanguBot hurt my Google rankings?
No. PanguBot is separate from Googlebot, which handles Google Search indexing. Blocking PanguBot has no effect on your traditional search rankings.
How can I tell if PanguBot is crawling my site?
Search your server access logs for the string "PanguBot". Our free AI Bot Log Analyzer does this in your browser: paste a log file and it counts hits per AI crawler, including PanguBot, with per-path breakdowns.
More Huawei agents
Part of our directory of every known AI crawler, refreshed monthly. Last verified: 2026-07-18.