BulkAudits fetches public pages the way a polite crawler does.

When someone audits a site, BulkAudits requests its home page, robots.txt, sitemap.xml, llms.txt and a sample of internal pages, from Vercel infrastructure, with this user agent:

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36 BulkAudits/2.0 (+https://www.bulkaudits.com/bot)

One additional request identifies as Googlebot to detect hosting rules that reject search crawlers. BulkAudits never identifies as the crawler of an AI vendor: robots.txt is evaluated for those tokens by reading the file, not by impersonating GPTBot, ClaudeBot or PerplexityBot.

Requests are capped per audit (at most the page limit of a plan, a few hundred pages), run with a small concurrency, and stop when a site returns errors. Page bodies are read to produce findings and are not kept beyond the audit; the report stores URLs, headers and extracted metadata.

To keep BulkAudits off a site, add a robots.txt group for BulkAudits with Disallow: /. Audits of a site are started by people who enter its address; if a site of yours is audited by someone else and you object, email polostudio.brand@gmail.com.

AI bot catalog version 2026-09-16. See AI Search Readiness for what is checked.