ev-crawler

Operated by Headline. Last seen today.

What is ev-crawler?

About

ev-crawler is an intelligence gatherer operated by Headline. It's not currently classified as artificially intelligent or AI-related in any way. If you think this is incorrect or want to provide more detail, please contact us.

Detail

Website https://headline.com/legal/crawler

Type

Intelligence Gatherer
Searches for useful insights

Expected Behavior

The behavior of Intelligence Gatherers depends on the goals of their clients. For example, a client might be interested in brand sentiment, in which case the agent would crawl related social media or blog posts at a more frequent rate than unrelated websites.

Access Control

Using Robots.txt

User Agent Token Description
ev-crawler Should match instances of ev-crawler

You can block ev-crawler or limit its access by setting user agent token rules in your website's robots.txt.

# robots.txt
# This blocks ev-crawler

User-agent: ev-crawler
Disallow: /

Instead of editing your robots.txt manually, use the API or Wordpress plugin to keep your robots.txt up to date with the agent list automatically.

Get the Robots.txt

Insights

Other Websites

0%
of top websites are currently blocking ev-crawler in some way
Updated today