Magnet.me-web

What is Magnet.me-web?

About

Magnet.me-web crawler monitors customer company websites to automatically synchronize and update job opening listings for the Magnet.me recruitment platform. You can see how often Magnet.me-web visits your website by setting up Dark Visitors Agent Analytics.

Expected Behavior

Fetchers visit websites on-demand. They're normally dispatched by apps to present metadata about a link (e.g. the title and thumbnail image) to a user.

Type

Fetcher

Fetches web content on behalf of an app

Detail

Operated By	Magnet.me
Last Updated	3 hours ago

Insights

Top Website Robots.txts

0% of top websites are blocking Magnet.me-web

Learn How →

Country of Origin

Unknown

Magnet.me-web has no known country of origin

Global Traffic

See All Global Bot & LLM Insights →

The percentage of all internet traffic coming from Fetchers

How Do I Get These Insights for My Website?

Use the WordPress plugin, Node.js package, or API to get started in seconds.

Set Up Agent Analytics

Robots.txt

Should I Block Magnet.me-web?

Probably not. If you block a fetcher, it won't be able to fetch the metadata it needs to display a link preview. Fewer people will click on the link to your website without this.

How Do I Block Magnet.me-web?

You can block Magnet.me-web or limit its access by setting user agent token rules in your website's robots.txt. Set up Dark Visitors Agent Analytics to check whether it's actually following them.

How Do I Block All Fetchers?

Serve a continuously updating robots.txt that blocks new fetchers automatically.

Set Up Automatic Robots.txt

# In your robots.txt ...

User-agent: Magnet.me-web # https://darkvisitors.com/agents/magnet-me-web
Disallow: /

⚠️ Manual Robots.txt Editing Is Not Scalable

New agents are created every day. We recommend setting up Dark Visitors Automatic Robots.txt if you want to block all agents of this type.