SiteCheckerBotCrawler
Last seen referenced today.
What is SiteCheckerBotCrawler?
About
SiteCheckerBotCrawler is an uncategorized agent. It's not currently known to be artificially intelligent or AI-related. If you think that's incorrect or can provide more detail about its purpose, please contact us.
Type
Uncategorized
Not currently assigned a type
Expected Behavior
Behavior will vary depending on whether this agent is a search engine crawler, data scraper, archiver, one-off fetcher, etc.
Insights
Other Websites
Access Control
Using Robots.txt
User Agent Token | Description |
---|---|
SiteCheckerBotCrawler |
Should match instances of SiteCheckerBotCrawler |
You can block SiteCheckerBotCrawler or limit its access by setting user agent token rules in your website's robots.txt.
# robots.txt
# This blocks SiteCheckerBotCrawler
User-agent: SiteCheckerBotCrawler
Disallow: /