SiteAuditBot

Operated by Semrush. Last seen today.

What is SiteAuditBot?

About

SiteAuditBot is a developer helper operated by Semrush. It's not currently known to be artificially intelligent or AI-related. If you think that's incorrect or can provide more detail about its purpose, please contact us.

Detail

Website

http://semrush.com/bot.html

Type

Developer Helper

Used by developers to test website functionality

Expected Behavior

Developer Helpers generally visit websites on direct behalf of a user who is evaluating some feature or attribute. Visits can be one-off or part of a larger crawl.

Insights

Other Websites

of top websites are currently blocking SiteAuditBot in some way

Learn How →

Access Control

Should I Block SiteAuditBot?

Probably not. Developer helpers are normally used to optimize or find problems with your website.

Using Robots.txt

User Agent Token	Description
`SiteAuditBot`	Should match instances of SiteAuditBot

You can block SiteAuditBot or limit its access by setting user agent token rules in your website's robots.txt.

# robots.txt
# This blocks SiteAuditBot

User-agent: SiteAuditBot
Disallow: /

Instead of doing this manually, you can generate your robots.txt automatically using the free API or Wordpress plugin.

Get the Robots.txt