YandexBot
Yandex's search / answer indexing crawler.
Quick answer
YandexBot is Yandex's main indexing robot, the crawler responsible for building the index behind Yandex search results. It's distinct from Yandex's other, more specialized robots, and it's the one most site owners mean when they talk about blocking Yandex entirely from a site.
What it does
YandexBot is documented as the main indexing robot Yandex uses, meaning it carries the core job of discovering and collecting pages for Yandex's search index, similar to what Googlebot does for Google Search. Yandex operates a number of more specialized robots for other functions, and its own compliance documentation notes that a robots.txt rule addressed specifically to 'User-agent: YandexBot' is used only by this main indexing bot, not by those other, more specialized robots, which can have different rules of their own. Yandex's support documentation describes YandexBot as taking the general robots.txt rules into account, which it distinguishes from the behavior of some of its other crawlers. A site that wants to control Yandex's overall indexing specifically should address its rule to YandexBot.
Facts
Operator
Yandex
Robots token
YandexBot
User agent string(s)
- Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)
Purpose
Search / answer indexing
Respects robots.txt
Yes
From Yandex's documentation, checked Oct 2026.
Should you block it?
Disallowing YandexBot removes a site from Yandex's organic search index over time, which matters mainly to a site with an audience in regions or markets where Yandex search has real usage, since Yandex isn't widely used in many parts of the world. A site with little or no Russian-language or Eastern European audience may see limited practical benefit from Yandex traffic either way, making this a lower-priority decision than blocking Googlebot or Bingbot for most Western-focused sites. Because a rule addressed to 'User-agent: YandexBot' only controls Yandex's main indexing robot and not its other, more specialized robots, a site wanting to fully opt out of all Yandex crawling would need to check Yandex's documentation for those other bots too rather than assuming one rule covers everything. This decision is about search visibility in Yandex's specific markets, not about AI training or AI answer visibility.
How to block it
Add this to robots.txt to block YandexBot from crawling any page: User-agent: YandexBot Disallow: /*id= A rule addressed to 'User-agent: YandexBot' controls only Yandex's main indexing robot; Yandex's other, more specialized robots are controlled separately.
Frequently asked questions
What does YandexBot do?
Yandex's own documentation describes the robot identified by the YandexBot user agent as its main indexing robot, the one behind Yandex search results.
Does a YandexBot rule in robots.txt control all of Yandex's crawlers?
No. Yandex says a 'User-agent: YandexBot' rule is used only by its main indexing bot, not by its other, more specialized robots.
Is blocking YandexBot worth it for a site with no Russian-speaking audience?
It depends on the site's actual audience; Yandex search has limited reach in many regions, so the practical impact varies by where a site's visitors are.
Related
- AI crawler directory — every AI crawler's robots.txt token, purpose and facts in one place.
- Free AI crawler checker — check which AI crawlers a domain's robots.txt actually allows or blocks.
- Glossary — definitions of the AI-visibility terms that come up alongside crawler behavior.
Sources
- The robot identified by the YandexBot/3.0 user agent is Yandex's main indexing robot. Source: https://yandex.com/support/webmaster/en/robot-workings/check-yandex-robots.md (checked Oct 2026)
- A 'User-agent: YandexBot' rule in robots.txt is used only by Yandex's main indexing bot. Source: https://yandex.com/support/webmaster/en/robot-workings/user-agent.md (checked Oct 2026)
MarketHQ tracks brand mentions across communities, news, blogs, social and AI answers, and turns them into gap analysis and action plans.