Loading
AI crawler comparison
Applebot powers Siri and Spotlight; Applebot-Extended is purely the Apple Intelligence training opt-out.
Short answer
They are not interchangeable. Applebot handles AI search and live answers, Applebot-Extended handles AI model training. Blocking one has no effect on the other, so a robots.txt rule aimed at the wrong agent produces exactly the outcome you were trying to avoid.
| Applebot | Applebot-Extended | |
|---|---|---|
| Operator | Apple | Apple |
| Purpose | AI search & live answers | AI model training |
| robots.txt | Respects robots.txt | Respects robots.txt |
| Published IP ranges | Yes — verifiable | Yes — verifiable |
| Official docs | Yes | Yes |
| Blocking costs you | AI-answer citations | No traffic — training only |
Applebot is Apple's standard web crawler, feeding Siri answers and Spotlight web suggestions. It's the crawler that physically fetches pages for Apple — the separate Applebot-Extended token then governs whether that fetched content may also train Apple Intelligence models. Blocking Applebot itself removes you from Siri and Spotlight results.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: Applebot.
Applebot-Extended, like Google-Extended, is a robots.txt control token rather than a distinct crawler: pages are still fetched by the regular Applebot that powers Siri and Spotlight. Disallowing Applebot-Extended opts your content out of Apple Intelligence model training without affecting Siri/Spotlight visibility.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: Applebot-Extended.
User-agent: Applebot Disallow: / User-agent: Applebot-Extended Disallow: /
The split most sites want when the two agents do different jobs: opt out of one without giving up the other.
User-agent: Applebot Disallow: / User-agent: Applebot-Extended Allow: /
Applebot is Apple's AI search and live answers agent — Apple's crawler powering Siri and Spotlight suggestions. Applebot-Extended is Apple's AI model training agent — Controls use of content for training Apple's AI models (Apple Intelligence). They are separate user agents with separate robots.txt rules.
No. robots.txt matches on the user-agent token, so a group naming Applebot applies only to Applebot. Applebot-Extended reads the group that names it, or the wildcard group if none does. To stop both you need a rule for each — or a wildcard rule, which would also affect every other crawler that reads it.
It depends which outcome you want. Blocking Applebot removes your pages from Apple's cited AI answers — that is a visibility loss, not a privacy win. Blocking Applebot-Extended opts you out of Apple's training data on the same terms. The common choice is to block training agents and allow search agents, so your content stays citable without feeding model training.
Applebot: Apple documents Applebot thoroughly and honors robots.txt directives. Applebot-Extended: Apple documents the token officially and honors it via Applebot's crawl infrastructure.
Yes — they send different user-agent strings, so grep for each token separately: grep -i "Applebot" and grep -i "Applebot-Extended". Be aware that a user-agent header is self-declared and anything can send either string. For a check that cannot be spoofed, match the request IP against the published ranges: Applebot at https://search.developer.apple.com/applebot.json, Applebot-Extended at https://search.developer.apple.com/applebot.json.
Part of our directory of every known AI crawler. Last verified: 2026-09-13.