AI crawler comparison
Applebot vs Applebot-Extended: what's the difference?
Applebot powers Siri and Spotlight; Applebot-Extended is purely the Apple Intelligence training opt-out.
Short answer
They are not interchangeable. Applebot handles AI search and live answers, Applebot-Extended handles AI model training. Blocking one has no effect on the other, so a robots.txt rule aimed at the wrong agent produces exactly the outcome you were trying to avoid.
Side by side
| Applebot | Applebot-Extended | |
|---|---|---|
| Operator | Apple | Apple |
| Purpose | AI search & live answers | AI model training |
| robots.txt | Respects robots.txt | Respects robots.txt |
| Published IP ranges | Yes — verifiable | Yes — verifiable |
| Official docs | Yes | Yes |
| Blocking costs you | AI-answer citations | No traffic — training only |
What Applebot does
Applebot is Apple's standard web crawler, feeding Siri answers and Spotlight web suggestions. It's the crawler that physically fetches pages for Apple — the separate Applebot-Extended token then governs whether that fetched content may also train Apple Intelligence models. Blocking Applebot itself removes you from Siri and Spotlight results.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: Applebot.
What Applebot-Extended does
Applebot-Extended, like Google-Extended, is a robots.txt control token rather than a distinct crawler: pages are still fetched by the regular Applebot that powers Siri and Spotlight. Disallowing Applebot-Extended opts your content out of Apple Intelligence model training without affecting Siri/Spotlight visibility.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: Applebot-Extended.
Block both
User-agent: Applebot Disallow: / User-agent: Applebot-Extended Disallow: /
Block Applebot, allow Applebot-Extended
The split most sites want when the two agents do different jobs: opt out of one without giving up the other.
User-agent: Applebot Disallow: / User-agent: Applebot-Extended Allow: /
Check your own site
- AI Crawler Access Checker — see which of the two your live robots.txt currently allows.
- robots.txt Generator for AI Bots — build per-bot rules for both agents and the other 26.
- AI Bot Log Analyzer — paste your access log and see which of them is actually visiting.
- Should you block AI bots? — the decision framework behind the split above.
Frequently asked questions
What is the difference between Applebot and Applebot-Extended?
Applebot is Apple's AI search and live answers agent — Apple's crawler powering Siri and Spotlight suggestions. Applebot-Extended is Apple's AI model training agent — Controls use of content for training Apple's AI models (Apple Intelligence). They are separate user agents with separate robots.txt rules.
Does blocking Applebot also block Applebot-Extended?
No. robots.txt matches on the user-agent token, so a group naming Applebot applies only to Applebot. Applebot-Extended reads the group that names it, or the wildcard group if none does. To stop both you need a rule for each — or a wildcard rule, which would also affect every other crawler that reads it.
Should I block Applebot, Applebot-Extended, or both?
It depends which outcome you want. Blocking Applebot removes your pages from Apple's cited AI answers — that is a visibility loss, not a privacy win. Blocking Applebot-Extended opts you out of Apple's training data on the same terms. The common choice is to block training agents and allow search agents, so your content stays citable without feeding model training.
Do Applebot and Applebot-Extended both respect robots.txt?
Applebot: Apple documents Applebot thoroughly and honors robots.txt directives. Applebot-Extended: Apple documents the token officially and honors it via Applebot's crawl infrastructure.
Can I tell Applebot and Applebot-Extended apart in my server logs?
Yes — they send different user-agent strings, so grep for each token separately: grep -i "Applebot" and grep -i "Applebot-Extended". Be aware that a user-agent header is self-declared and anything can send either string. For a check that cannot be spoofed, match the request IP against the published ranges: Applebot at https://search.developer.apple.com/applebot.json, Applebot-Extended at https://search.developer.apple.com/applebot.json.
Other comparisons
Part of our directory of every known AI crawler. Last verified: 2026-08-14.