Google-Extended

Google-Extended is a robots.txt product token that controls whether Google uses crawled content for Gemini and Vertex grounding. It is not a distinct crawler that shows up in access logs as Google-Extended.

Operator: Google
Motif family: google_ai
Motif class: robots.txt product token
In Motif classifier: No
robots.txt: This token exists for robots.txt control, not as a crawl User-Agent
Executes JavaScript: Not applicable
Verified in Motif: Not an HTTP User-Agent Motif can match

What Motif will and will not claim

Motif's classifier intentionally omits Google-Extended. Robots policy-only tokens are not HTTP User-Agents, so a tracker can never observe them. Directories that list Google-Extended beside GPTBot are mixing two different objects.

Revenue

Changing Google-Extended does not create or remove gemini.google.com visitors. Gemini first touch is still a person.

Vendor documentation

Start with the vendor page at https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers. Motif stores the controlled token from a self-reported User-Agent.

Google-Extended User-Agent example

Not a User-Agent. Place User-agent: Google-Extended in robots.txt.

Allow Google-Extended in robots.txt

User-agent: Google-Extended
Allow: /

Block Google-Extended in robots.txt

User-agent: Google-Extended
Disallow: /

Does Google-Extended count as a visit in Motif?

Google-Extended is not an HTTP User-Agent, so Motif never records it as a crawler hit or a visit.

Will Motif show Google-Extended on my HTML pages?

Only if that client requested /m.js or POST /v1/events. HTML-only crawls of the origin never reach Motif. Saying otherwise would be a product lie.

Should I block Google-Extended?

This is a robots.txt product token. Allow or disallow it according to whether you want that vendor to use Applebot or Googlebot-crawled content for the named product. Motif will not observe the token either way.