AI crawler comparison
Google-Extended vs GoogleOther: what's the difference?
Neither is Googlebot, and only one of them is an AI training control.
Short answer
They are not interchangeable. Google-Extended handles AI model training, GoogleOther handles both training and AI search. Blocking one has no effect on the other, so a robots.txt rule aimed at the wrong agent produces exactly the outcome you were trying to avoid.
Side by side
| Google-Extended | GoogleOther | |
|---|---|---|
| Operator | ||
| Purpose | AI model training | Training + search |
| robots.txt | Respects robots.txt | Respects robots.txt |
| Published IP ranges | Yes — verifiable | Yes — verifiable |
| Official docs | Yes | Yes |
| Blocking costs you | No traffic — training only | AI-answer citations |
What Google-Extended does
Google-Extended is not a separate crawler — it is a robots.txt control token evaluated by Google's existing crawl infrastructure. Disallowing it opts your content out of Gemini training and grounding without affecting Google Search indexing or ranking, which remain governed by Googlebot.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: Google-Extended.
What GoogleOther does
GoogleOther is Google's catch-all crawler for uses outside Search indexing — research, internal development, and product one-offs, which can include AI-related work. It exists so those fetches don't masquerade as Googlebot. Blocking it does not affect Google Search rankings, which remain governed by Googlebot.
Full profile, with every robots.txt, nginx, Apache and Cloudflare rule: GoogleOther.
Block both
User-agent: Google-Extended Disallow: / User-agent: GoogleOther Disallow: /
Block Google-Extended, allow GoogleOther
The split most sites want when the two agents do different jobs: opt out of one without giving up the other.
User-agent: Google-Extended Disallow: / User-agent: GoogleOther Allow: /
Check your own site
- AI Crawler Access Checker — see which of the two your live robots.txt currently allows.
- robots.txt Generator for AI Bots — build per-bot rules for both agents and the other 26.
- AI Bot Log Analyzer — paste your access log and see which of them is actually visiting.
- Should you block AI bots? — the decision framework behind the split above.
Frequently asked questions
What is the difference between Google-Extended and GoogleOther?
Google-Extended is Google's AI model training agent — Controls use of content for Gemini and Vertex AI training, separate from Googlebot. GoogleOther is Google's both training and AI search agent — Google's generic crawler for research and internal product uses. They are separate user agents with separate robots.txt rules.
Does blocking Google-Extended also block GoogleOther?
No. robots.txt matches on the user-agent token, so a group naming Google-Extended applies only to Google-Extended. GoogleOther reads the group that names it, or the wildcard group if none does. To stop both you need a rule for each — or a wildcard rule, which would also affect every other crawler that reads it.
Should I block Google-Extended, GoogleOther, or both?
It depends which outcome you want. Blocking Google-Extended opts you out of Google's model training and costs you no traffic today, because training crawlers send no visitors. Blocking GoogleOther removes you from the answers Google's assistant shows its users. The common choice is to block training agents and allow search agents, so your content stays citable without feeding model training.
Do Google-Extended and GoogleOther both respect robots.txt?
Google-Extended: As a robots.txt token rather than a crawler, honoring your directive is its entire function; Google documents it officially. GoogleOther: Officially documented by Google and respects robots.txt like the rest of its crawler family.
Can I tell Google-Extended and GoogleOther apart in my server logs?
Yes — they send different user-agent strings, so grep for each token separately: grep -i "Google-Extended" and grep -i "GoogleOther". Be aware that a user-agent header is self-declared and anything can send either string. For a check that cannot be spoofed, match the request IP against the published ranges: Google-Extended at https://developers.google.com/static/crawling/ipranges/common-crawlers.json, GoogleOther at https://developers.google.com/static/crawling/ipranges/common-crawlers.json.
Other comparisons
Part of our directory of every known AI crawler. Last verified: 2026-08-14.