Anthropic · AI answers

Claude-User

Visits a page when someone asks Claude a question that needs it. Blocking it stops Claude from reading your pages for user questions, which can reduce your visibility in answers.

robots.txt name

Claude-User

User agent

Claude-User

Category

AI answers: Fetch a page when someone asks an assistant about it.

Follows robots.txt

Yes

Official documentation

https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler

Published IP ranges

https://claude.com/crawling/bots.json

Checked against the official documentation on October 17, 2026.

Block or allow it in robots.txt

Add these lines to the robots.txt file at the root of your domain. To block it everywhere:

User-agent: Claude-User
Disallow: /

To allow it everywhere except one folder:

User-agent: Claude-User
Disallow: /private/

Is it really this bot?

Anyone can copy a user agent. To be sure a request comes from Anthropic, compare its IP address with the ranges the company publishes.

Other bots from Anthropic

• Claude-SearchBot · Search indexes
• ClaudeBot · Training crawlers