Set Up Crawler & Scraper Analytics

Installing the WordPress plugin is easy and only takes a few seconds.

• Log in to your website's WordPress dashboard
• Click Plugins in the sidebar
• Search for "Dark Visitors" or download the plugin directly
• Click Install Now
• Click Activate
• Click Dark Visitors in the sidebar
• Paste your access token
• Click Save Changes

The Node.js package and docs are available on NPM.

Call the Visits API to track each incoming pageview request to your website.

The Request

Endpoint
URL	https://api.darkvisitors.com/visits
HTTP Method	POST
Headers
Authorization	A bearer token with your project's access token (e.g. Bearer 48d7dcbd-fc44-4b30-916b-2a5955c8ee42).
Content-Type	This needs to be set to application/json
Body
request_path	The URL path of your visitor's request
request_method	The HTTP method of your visitor's request (e.g. GET, POST, etc.)
request_headers	The HTTP headers of your visitor's request, as a key-value object

Tips

• Add this call in middleware to cover all of your pages from one place.
• Don't wait for the API response (i.e. make it non-blocking) to avoid adding latency to your page response.
• Ensure that errors are handled in a way that does not make your page response fail too.

Example

This cURL example sends a visit. If the visit is from an artificial agent, it will show up in agent analytics.

curl -X POST https://api.darkvisitors.com/visits \
-H "Authorization: Bearer ${ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
    "request_path": "'"${request.path}"'",
    "request_method": "'"${request.method}"'",
    "request_headers": "'"${request.headers}"'"
}'