Meta-ExternalFetcher
Last updated 8 hours ago.
What is Meta-ExternalFetcher?
About
Meta-ExternalFetcher is dispatched by Meta AI products in response to user prompts, when they need to fetch an individual links.
Detail
Operator | Meta |
Documentation | https://developers.facebook.com/docs/sharing/webmasters/crawler |
Type
AI Assistant
Intelligently performs tasks on behalf of a user
Expected Behavior
AI assistants generally make one-off visits to websites based on user requests, rather than crawling the web in any automatic fashion.
Insights
Activity on Your Website
Other Websites
Access Control
Should I Block Meta-ExternalFetcher?
Probably not. AI assistants visit websites directly on behalf of human users, so blocking them will effectively block those users. This could lead to a poor user experience and possible negative sentiment about your website. Not blocking AI assistants will allow more human users to use your website as they choose.
Using Robots.txt
User Agent Token | Description |
---|---|
Meta-ExternalFetcher |
Should match instances of Meta-ExternalFetcher |
You can block Meta-ExternalFetcher or limit its access by setting user agent token rules in your website's robots.txt.
# robots.txt
# This should block Meta-ExternalFetcher
User-agent: Meta-ExternalFetcher
Disallow: /