Perplexity-User

What it does

Perplexity-User fetches a page specifically because a user asked Perplexity a question that required visiting that URL, rather than as part of any ongoing or scheduled crawl. Perplexity documents it as supporting user actions, explicitly stating it is not used for web crawling or to collect content for training AI foundation models, which separates it clearly from a bot built for either of those jobs. Because the fetch is triggered by a real person's request, Perplexity says this fetcher generally ignores robots.txt rules, reasoning that a disallow rule aimed at automated crawling doesn't cleanly apply to a one-off, user-initiated lookup. Perplexity publishes this bot's IP ranges separately, so a site owner can confirm genuine traffic even though robots.txt itself may not reliably control it.

Facts

Should you block it?

Because Perplexity says Perplexity-User generally ignores robots.txt rules, a disallow rule for this bot is more of a stated preference than a reliable block, which changes the usual calculus: a brand can't simply assume a robots.txt rule keeps Perplexity from fetching a page when a user asks about it directly. For a brand that wants to be described accurately when someone asks Perplexity about it, that's arguably a benefit, since the fetch happens specifically to answer a real question rather than for bulk training or indexing. MarketHQ tracks how brands appear in Perplexity's answers, and a site's content being reachable through Perplexity-User is part of what lets Perplexity ground an answer in current, first-party information rather than relying only on what it may have learned elsewhere. There's little practical upside to attempting to block a fetcher that the operator itself says mostly ignores that attempt.

How to block it

Add this to robots.txt to state a preference against Perplexity-User fetches: User-agent: Perplexity-User Disallow: / Perplexity says this fetcher generally ignores robots.txt rules since the request is user-initiated, so this is a stated preference rather than a guaranteed block.

Related

• AI crawler directory — every AI crawler's robots.txt token, purpose and facts in one place.
• Free AI crawler checker — check which AI crawlers a domain's robots.txt actually allows or blocks.
• Glossary — definitions of the AI-visibility terms that come up alongside crawler behavior.