Google

GoogleAgent-Mariner is an AI agent created by Google that can use a web browser. It can intelligently navigate and interact with websites to complete multi-step tasks on behalf of a human user.

Gemini-Deep-Research is the agent responsible for collecting and scanning resources used in Google Gemini's Deep Research feature, which acts as a personal research assistant.

Google-NotebookLM is an AI-powered research and note-taking assistant that helps users synthesize information from their own uploaded sources, such as documents, transcripts, or web content. It can generate summaries, answer questions, and highlight key themes from the materials you provide, acting like a personalized research companion built on Google's Gemini model. Google-NotebookLM fetches source URLs when users add them to their notebooks, enabling the AI to access and analyze those pages for context and insights.

CloudVertexBot is a Google-operated crawler available to site owners to request targeted crawls of their own sites for AI training purposes on the Vertex AI platform.

Google-Extended is a web crawler used by Google to download AI training content for its AI products like the Gemini assistant and its Vertex AI generative APIs.

GoogleOther is Google's generic crawler used by various product teams for fetching publicly accessible content, including one-off crawls for internal research and development.

Chrome-Lighthouse is an automated, open-source tool that audits web pages for performance, accessibility, progressive web apps, SEO, and best practices. It runs a series of audits against a page and generates a comprehensive report on how well the page did.

Google Inspection Tool is a developer helper operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Schema Markup Testing Tool is a developer helper operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Trust Services (DCV Check) is a developer helper operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google-Site-Verification is a verification service that proves website ownership for Google Search Console access by checking verification tokens like HTML tags or files on websites.

GoogleAssociationService is a verification bot that checks Digital Asset Links statements on websites to validate relationships between web properties and mobile apps for features like app linking and credential sharing.

ProjectShield Url Check is a developer helper operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Feed Fetcher is a fetcher operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Image Proxy is a fetcher operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Publisher Center is a fetcher operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Web Preview is a fetcher operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google-AdWords-Express is an automated crawler that analyzes advertiser websites to assist with Google Ads creation and campaign optimization, specifically designed for small business advertising needs and site verification.

Google-PageRenderer is a specialized tool used by Google services for rendering web pages and fetching content when triggered by end-user actions, supporting various Google products and functionalities that require real-time page processing.

Google-Read-Aloud fetches web content to provide text-to-speech functionality, enabling users to have web pages read aloud through Google's accessibility services.

GoogleImageProxy fetches and caches images and link previews for Gmail email rendering, enabling safe display of external content within Google's email interface.

GoogleProducer is a specialized Google crawler that fetches and processes news feeds and content that publishers explicitly provide for display on Google News landing pages and related Google News services.

AdsBot-Google is a specialized Google crawler that monitors and analyzes websites where Google Ads are served to ensure quality and policy compliance.

AdsBot-Google-Mobile is an intelligence gatherer operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google AdsBot is an intelligence gatherer operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google AdSense is an intelligence gatherer operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google-Adwords-Instant is the web crawler bot connected to the Google Ads platform that visits advertiser landing pages to perform quality checks, verification, and compliance assessment for advertising campaigns.

Mediapartners-Google crawls website content to analyze page context and keywords for Google AdSense ad targeting, determining appropriate advertisements to display based on site content and user demographics.

FeedFetcher-Google is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Favicon is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Images is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Scholar is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google Videos is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Googlebot is Google's web crawler that discovers and indexes web content for Google Search, including both mobile and desktop variants that crawl websites to understand their content.

Googlebot-IA is Google Scholar's specialized crawler that indexes scholarly literature from academic publishers, repositories, and university websites to populate the academic search engine with research papers and publications.

Googlebot-Image is Google's specialized search engine crawler that discovers and indexes images across the web for inclusion in Google Images search results, analyzing visual content and metadata.

Googlebot-Mobile is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Googlebot-News is a search engine crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Googlebot-Video is Google's specialized crawler for video content discovery and indexing. It crawls websites to find and analyze video files, metadata, and thumbnails for Google Video Search results.

Google-InspectionTool powers Search Console's URL Inspection tool and Rich Results Test, crawling pages to validate indexability, structured data markup, and search feature compatibility on user request.

Google-Safety is a specialized security crawler that performs abuse-specific scanning to detect malware and other security threats on publicly posted links across Google properties, operating independently of robots.txt restrictions.

Google-Trust-Services is a certificate authority bot that helps build a safer Internet by providing transparent, trusted, and reliable TLS certificates. It crawls websites to verify certificate authenticity and manage secure connections for Google services and users.

ProjectShield-UrlCheck is a security scanner that monitors websites protected by Google's Project Shield service, checking for malicious traffic and DDoS attacks to ensure continuous protection.

Google Digital Asset Links is an SEO crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google StoreBot is an SEO crawler operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.

Google-Structured-Data-Testing bot validates structured data markup on web pages, helping developers ensure proper schema implementation for rich search results and SEO optimization in Google Search.

Storebot-Google is Google's web crawler that analyzes e-commerce websites, product pages, and checkout processes using machine learning to gather information on pricing, delivery, payments and shopping experiences.

Google-CloudVertexBot is an uncategorized agent operated by Google. If you think this is incorrect or can provide additional detail about its purpose, please let us know.