Skip to content

Meta · AI answers

Meta-ExternalFetcher

Fetches individual links at a user's request and supports Meta's agentic AI features, such as helping AI navigate websites to complete tasks.

robots.txt name
meta-externalfetcher
User agent
meta-externalfetcher/1.1
Category
AI answers: Fetch a page when someone asks an assistant about it.
Follows robots.txt
Partly. Meta says it may bypass robots.txt because it performs fetches requested by the user.
Official documentation
https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/

Checked against the official documentation on October 17, 2026.

Block or allow it in robots.txt

Add these lines to the robots.txt file at the root of your domain. To block it everywhere:

User-agent: meta-externalfetcher
Disallow: /

To allow it everywhere except one folder:

User-agent: meta-externalfetcher
Disallow: /private/

This bot does not always follow robots.txt. To keep it out for sure, block it on your server or firewall by user agent and, where published, by IP range.

Is it really this bot?

Anyone can copy a user agent, and Meta does not publish IP ranges for this bot. A reverse DNS lookup of the IP address may help; check the official documentation.

Other bots from Meta

See which AI crawlers read your site

Statsdot keeps bots out of your visitors and, with AI crawler tracking, shows which crawlers read your pages and which pages they request. No cookies, no personal data.

Try Statsdot free for 30 days How AI crawler tracking works