Applebot-Extended
Apple's ai model training crawler.
Quick answer
Applebot-Extended is a robots.txt-only control from Apple that governs whether content already crawled by Applebot can be used to train Apple's generative AI foundation models. It doesn't crawl anything itself; it's a permission flag layered on top of Applebot's existing crawl.
What it does
Applebot-Extended doesn't crawl web pages on its own. Instead, it's a robots.txt token that controls what Apple is allowed to do with content Applebot has already fetched, specifically whether that content can be used to train Apple's generative AI foundation models. A site can allow Applebot to crawl normally for Apple search features like Siri and Spotlight, while separately disallowing Applebot-Extended to opt that same content out of AI training. Because Applebot-Extended has no crawling function of its own, disallowing it doesn't remove a page from Apple's search index or stop Applebot from visiting; pages that disallow Applebot-Extended can still appear in Apple's search results, since that permission is handled independently from training use.
Facts
Operator
Apple
Robots token
Applebot-Extended
User agent string(s)
- (no separate HTTP user agent; controlled only via the Applebot-Extended robots.txt token)
Purpose
AI model training
Respects robots.txt
Yes
IP ranges
http://search.developer.apple.com/applebot.json
From Apple's documentation, checked Oct 2026.
Should you block it?
Disallowing Applebot-Extended is a narrow, low-risk choice for a site that wants to keep its content out of Apple's generative AI training data without giving up anything else. Because this token only controls training use and doesn't crawl or index anything itself, blocking it has no effect on whether Applebot can still crawl the site for Apple search, Siri, or Spotlight results, so there's little tradeoff on the search-visibility side. The more relevant question for a brand focused on AI visibility is whether Apple's generative AI products eventually surface brand mentions or recommendations the way other AI assistants do; since Apple's own documentation ties Applebot-Extended specifically to training rather than to live answers, blocking it is more about data control than about disappearing from, or appearing in, any AI assistant's immediate answers.
How to block it
Add this to robots.txt to opt content out of training Apple's generative AI models, without affecting Apple search crawling: User-agent: Applebot-Extended Disallow: / Applebot itself keeps crawling for search unless a separate rule disallows it too.
Frequently asked questions
Does Applebot-Extended crawl my site?
No. Apple says Applebot-Extended does not crawl webpages. It only controls how content already crawled by Applebot can be used for AI training.
Will disallowing Applebot-Extended remove my pages from Apple search results?
No. Apple says webpages that disallow Applebot-Extended can still be included in search results, since that permission is separate from training use.
How do I opt out of Apple's AI training specifically?
Disallow the Applebot-Extended token in robots.txt. Apple documents this as how publishers opt out of having content used to train its generative foundation models.
Related
- AI crawler directory — every AI crawler's robots.txt token, purpose and facts in one place.
- Free AI crawler checker — check which AI crawlers a domain's robots.txt actually allows or blocks.
- Glossary — definitions of the AI-visibility terms that come up alongside crawler behavior.
Sources
- Applebot-Extended does not crawl webpages; it only controls how data already crawled by Applebot may be used for training. Source: https://support.apple.com/en-us/119829 (checked Oct 2026)
- Publishers can opt out of having content used to train Apple's generative AI foundation models by disallowing Applebot-Extended. Source: https://support.apple.com/en-us/119829 (checked Oct 2026)
MarketHQ tracks brand mentions across communities, news, blogs, social and AI answers, and turns them into gap analysis and action plans.