Guides › AI crawlers
How to block meta-externalagent in robots.txt
meta-externalagent is Meta's crawler for AI use cases such as training models and improving products.
Block meta-externalagent from your whole site
User-agent: meta-externalagent Disallow: /
Block only part of your site
User-agent: meta-externalagent Disallow: /blog/ Disallow: /docs/private/
Notes
This is a different agent from facebookexternalhit, which fetches link previews when someone shares your URL on Facebook, Instagram or WhatsApp. Blocking facebookexternalhit breaks your share previews, so block only meta-externalagent if your goal is to opt out of AI.
A crawler follows only the most specific group that matches its name. If you add a User-agent: meta-externalagent group, it stops reading your User-agent: * rules entirely. Repeat any rules you still want it to follow inside its own group.
robots.txt is a request, not an access control. Reputable crawlers honour it; to enforce a block, use your CDN or firewall. Changes are usually picked up within about a day, because crawlers cache robots.txt.
Check it
Run your domain through the checker. It shows whether meta-externalagent and 11 other AI crawlers are allowed or blocked, and which line decides it.