Claude-User
Anthropic's user-triggered fetch crawler.
Quick answer
Claude-User is Anthropic's agent for fetching web content when a person asks Claude a question that requires visiting a specific site. It's triggered by a real user's request, not by an automated background crawl, and it follows standard robots.txt rules.
What it does
Claude-User activates when someone using Claude asks a question that needs information from a particular website, so Anthropic's systems retrieve that page on the user's behalf in the moment. Anthropic documents this as supporting Claude AI users directly, distinct from Claude-SearchBot, which handles broader search-style answers, and from ClaudeBot, which crawls separately to gather potential training data. Claude-User follows standard robots.txt directives, so a site that disallows it is opting out of having its pages fetched this way when a user's question points to that site. Anthropic publishes IP ranges covering its bots at claude.com/crawling/bots.json, which lets a site owner verify that traffic claiming to be Claude-User is genuine before deciding how to treat it.
Facts
Operator
Anthropic
Robots token
Claude-User
User agent string(s)
- Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)
Purpose
User-triggered fetch
Respects robots.txt
Yes
IP ranges
https://claude.com/crawling/bots.json
From Anthropic's documentation, checked Oct 2026.
Should you block it?
Disabling Claude-User means Claude can no longer retrieve a site's pages when a person directly asks about that site, and Anthropic says this can reduce the site's visibility for user-directed web search inside Claude. For a brand that wants to be found and accurately described when someone asks Claude about it by name, allowing Claude-User keeps that path open; blocking it trades away that visibility for tighter control over who fetches the site's pages. Because Claude-User is separate from ClaudeBot's training crawl and from Claude-SearchBot's search-style fetches, blocking it specifically targets this one use case rather than every way Claude might interact with a site. MarketHQ tracks how brands appear across AI engines including Claude, as part of its wider AI visibility tracking.
How to block it
Add this to robots.txt to block Claude-User from fetching pages on a user's behalf: User-agent: Claude-User Disallow: / This is separate from ClaudeBot and Claude-SearchBot, which need their own rules if a site wants to block those too.
Frequently asked questions
Is Claude-User the same bot that trains Claude's models?
No. That's ClaudeBot, a separate crawler. Claude-User only fetches pages when a real person asks Claude a question that points to a specific site.
What happens if I block Claude-User?
Anthropic says disabling Claude-User can reduce a site's visibility for user-directed web search inside Claude, since Claude can no longer fetch that site's pages on request.
How can I verify traffic claiming to be Claude-User?
Anthropic publishes its bots' IP ranges at claude.com/crawling/bots.json, so a site can check incoming requests against that list before deciding how to treat them.
Related
- AI crawler directory — every AI crawler's robots.txt token, purpose and facts in one place.
- Free AI crawler checker — check which AI crawlers a domain's robots.txt actually allows or blocks.
- Glossary — definitions of the AI-visibility terms that come up alongside crawler behavior.
Sources
- Claude-User fires when a person asks Claude a question that requires visiting a specific website. Source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (checked Oct 2026)
- Disabling Claude-User can reduce a site's visibility for user-directed web search. Source: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler (checked Oct 2026)
MarketHQ tracks brand mentions across communities, news, blogs, social and AI answers, and turns them into gap analysis and action plans.