Meta · Training crawlers
Meta-ExternalAgent
Crawls the web for uses such as training Meta's foundation AI models or improving products by indexing content directly. Blocking it keeps your content out of that crawl.
- robots.txt name
meta-externalagent- User agent
meta-externalagent/1.1- Category
- Training crawlers: Collect public pages that may be used to train AI models.
- Follows robots.txt
- Yes
- Official documentation
- https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/
Checked against the official documentation on October 17, 2026.
Block or allow it in robots.txt
Add these lines to the robots.txt file at the root of your domain. To block it everywhere:
User-agent: meta-externalagent
Disallow: /To allow it everywhere except one folder:
User-agent: meta-externalagent
Disallow: /private/Is it really this bot?
Anyone can copy a user agent, and Meta does not publish IP ranges for this bot. A reverse DNS lookup of the IP address may help; check the official documentation.
Other bots from Meta
- Meta-ExternalFetcher · AI answers
- Meta-WebIndexer · Search indexes
See which AI crawlers read your site
Statsdot keeps bots out of your visitors and, with AI crawler tracking, shows which crawlers read your pages and which pages they request. No cookies, no personal data.