OAI-SearchBot

Last updated 5 hours ago.

What is OAI-SearchBot?

About

OAI-SearchBot is used to create an index of websites that can be surfaced as results in OpenAI's SearchGPT product.

Track OAI-SearchBot on Your Website
You can see when OAI-SearchBot visits your website using the API or WordPress plugin.
Set Up Agent Analytics

Detail

Operator OpenAI
Documentation https://platform.openai.com/docs/bots

Type

AI Search Crawler
Indexes web content for AI-powered search results

Expected Behavior

AI search crawlers do not adhere to a fixed visitation schedule for websites. The frequency of visits varies widely based on many factors, and can even happen on-demand in response to a user query.

Insights

OAI-SearchBot's Activity on Your Website

Half of your traffic probably comes from artificial agents, and there are more of them every day. Track their activity with the API or WordPress plugin.

Set Up Agent Analytics

Other Websites

3%
of top websites are currently blocking OAI-SearchBot in some way
Learn How →

Access Control

Should I Block OAI-SearchBot?

Probably not. AI search crawlers power AI search engines, which are a useful way for users to discover your website. AI search engine assistants usually cite their sources and provide links to their results.

Using Robots.txt

User Agent Token Description
OAI-SearchBot Should match instances of OAI-SearchBot

You can block OAI-SearchBot or limit its access by setting user agent token rules in your website's robots.txt.

# robots.txt
# This should block OAI-SearchBot

User-agent: OAI-SearchBot
Disallow: /

Instead of doing this manually, you can use the API or Wordpress plugin to keep your robots.txt updated with the latest known AI scrapers, crawlers, and assistants automatically.

Set Up Automatic Robots.txt