Google · Training crawlers
Google-Extended
A robots.txt token, not a separate crawler: it controls whether content Google crawls may be used to train future Gemini models and for grounding in Gemini Apps and Vertex AI. It does not affect inclusion or ranking in Google Search.
- robots.txt name
Google-Extended- User agent
Google-Extended- Category
- Training crawlers: Collect public pages that may be used to train AI models.
- Follows robots.txt
- Yes
- Official documentation
- https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers
Checked against the official documentation on October 17, 2026.
Block or allow it in robots.txt
Add these lines to the robots.txt file at the root of your domain. To block it everywhere:
User-agent: Google-Extended
Disallow: /To allow it everywhere except one folder:
User-agent: Google-Extended
Disallow: /private/Is it really this bot?
Anyone can copy a user agent, and Google does not publish IP ranges for this bot. A reverse DNS lookup of the IP address may help; check the official documentation.
Other bots from Google
- Gemini Notebook · AI answers
- Google-Agent · AI answers
- Googlebot · Search indexes
- Google-CloudVertexBot · Other AI bots
- GoogleOther · Other AI bots
See which AI crawlers read your site
Statsdot keeps bots out of your visitors and, with AI crawler tracking, shows which crawlers read your pages and which pages they request. No cookies, no personal data.