FacebookExternalHit
Meta's user-triggered fetch crawler.
Quick answer
FacebookExternalHit is Meta's crawler for link previews. When someone shares a URL on Facebook, Instagram, or Messenger, this bot fetches the page to build the preview card showing its title, description, and thumbnail image for that specific share, in real time.
What it does
FacebookExternalHit fetches a page whenever someone shares its URL on a Meta app like Facebook, Instagram, or Messenger, gathering the title, description, and thumbnail image needed to show a link preview card in that share. This fetch is triggered by the act of sharing rather than running as a continuous background crawl, which is why it is described as a user-triggered fetch rather than a search or training crawler. Meta states that FacebookExternalHit does not always respect robots.txt and may bypass it specifically when performing security or integrity checks, such as scanning for malware or malicious content, even on a page that otherwise disallows it. There is no published IP range list for this bot, so verifying its traffic independently is harder than with crawlers that publish one.
Facts
Operator
Meta
Robots token
facebookexternalhit
User agent string(s)
- facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
- facebookexternalhit/1.1
Purpose
User-triggered fetch
Respects robots.txt
No
From Meta's documentation, checked Oct 2026.
Should you block it?
Blocking FacebookExternalHit stops link previews from rendering when people share a site's URLs on Facebook, Instagram, or Messenger, so shared links would show up as plain text instead of a card with a title, description, and image. For most brands this is a meaningful loss, since link previews affect click-through on shared content across some of the largest social platforms. Unlike most crawlers covered here, Meta states that FacebookExternalHit does not always honor robots.txt and may bypass a disallow rule specifically for security or integrity checks, so a block is not guaranteed to stop every type of request from this bot. This bot has no direct relationship to the AI assistants MarketHQ tracks, so blocking it affects social share previews rather than AI answer visibility.
How to block it
Add this to robots.txt to try to block FacebookExternalHit: User-agent: facebookexternalhit Disallow: / Meta says this bot might still bypass robots.txt for security or integrity checks such as malware scanning, so a disallow rule is not guaranteed to stop every request.
Frequently asked questions
What happens if I block FacebookExternalHit?
Shared links to the site on Facebook, Instagram, or Messenger would lose their title, description, and thumbnail preview.
Does FacebookExternalHit always respect robots.txt?
No. Meta says it might bypass robots.txt when performing security or integrity checks like malware scanning.
Is FacebookExternalHit related to AI model training?
No. Meta describes it as a link preview crawler triggered by sharing, unrelated to its AI training crawlers.
Related
- AI crawler directory — every AI crawler's robots.txt token, purpose and facts in one place.
- Free AI crawler checker — check which AI crawlers a domain's robots.txt actually allows or blocks.
- Glossary — definitions of the AI-visibility terms that come up alongside crawler behavior.
Sources
- FacebookExternalHit crawls content shared on Meta's apps to generate link previews (title, description, thumbnail). Source: https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers (checked Oct 2026)
- FacebookExternalHit might bypass robots.txt when performing security or integrity checks such as malware scanning. Source: https://developers.facebook.com/documentation/sharing/webmasters/web-crawlers (checked Oct 2026)
MarketHQ tracks brand mentions across communities, news, blogs, social and AI answers, and turns them into gap analysis and action plans.