Serve an Automatic Robots.txt From Your WordPress Website

Overview

Use the WordPress plugin to serve an automatic robots.txt from your website in just a few seconds. Please contact us if you need help.

Step 1: Install the Plugin

• Navigate to the Projects page and open your project
• Copy your access token from the Settings page
• Log in to your website's WordPress dashboard
• Click Plugins in the sidebar
• Search for "Dark Visitors"
• Click Install Now
• Click Activate
• Click Dark Visitors in the sidebar
• Paste in your access token
• Select the agent types you want to block. Allowed categories include:
  • AI Agent
  • AI Assistant
  • AI Data Scraper
  • AI Search Crawler
  • Archiver
  • Developer Helper
  • Fetcher
  • Headless Agent
  • Intelligence Gatherer
  • Scraper
  • SEO Crawler
  • Search Engine Crawler
  • Security Scanner
  • Undocumented AI Agent
  • Uncategorized
• Click Save Changes

Step 2: Test Your Integration

If your website is correctly connected, you should see the new rules in your website's robots.txt.