ClaudeBot
Anthropic documents ClaudeBot as a crawler that collects public web content for model-related use. It is not a person on claude.ai.
- Operator
- Anthropic
- Motif family
- anthropic
- Motif class
- Model-related crawler
- In Motif classifier
- Yes
- robots.txt
- Vendor documents that it honors robots.txt
- Executes JavaScript
- No
- Verified in Motif
- Anthropic publishes crawler IP guidance. Motif does not check source IPs
What Motif will and will not claim
Motif classifies ClaudeBot as family anthropic, class model, when it hits /m.js or the collector. HTML-only fetches never appear. Motif does not label the hit 'training'.
Revenue
Paying customers from Claude are people whose first-touch referrer is claude.ai. ClaudeBot cannot win the recommendation.
Vendor documentation
Start with the vendor page at https://support.anthropic.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web. Motif stores the controlled token from a self-reported User-Agent.
ClaudeBot User-Agent example
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ClaudeBot/1.0; +claudebot@anthropic.com
Allow ClaudeBot in robots.txt
User-agent: ClaudeBot
Allow: /
Block ClaudeBot in robots.txt
User-agent: ClaudeBot
Disallow: /
Does ClaudeBot count as a visit in Motif?
No. If ClaudeBot hits the Motif tracker or collector it increments AI crawlers. It does not create a visitor and cannot win a recommendation.
Will Motif show ClaudeBot on my HTML pages?
Only if that client requested /m.js or POST /v1/events. HTML-only crawls of the origin never reach Motif. Saying otherwise would be a product lie.
Should I block ClaudeBot?
Model-related crawlers are a data-policy call. Allowing or blocking them does not create Stripe customers. Motif will not pretend a crawler hit is demand.
Related