AI Visibility: What It Actually Means, and How to Measure the Part You Control

Two Different Questions Wearing One Name

Ask ten people what "AI visibility" means and you get two answers.

The first is about mentions. Does ChatGPT bring up your brand when someone asks for the best tool in your category? Does Perplexity cite your page? Does a Google AI Overview quote your pricing? This is what most tools in the category measure. They run prompts on a schedule, record which brands and sources come back, and chart it over time.

The second is about access. When an AI system goes to your site, can it fetch the page? Does it receive readable text or an empty shell? Can it tell what your business is called and what you sell?

Both are real. Only one of them is under your control, and it is the one almost nobody checks.

You cannot make an assistant recommend you. You can make sure that when it reaches for your site, it finds something worth using. That is the difference between tracking a score and fixing a problem, and it is the reason this article is about the second question.

Nobody Is Blocking "AI". There Are Four Different Crawlers.

The single most common mistake we see is treating AI crawlers as one switch. They are not one thing, and they are controlled separately.

Search index crawlers build the index an assistant searches when it answers. OAI-SearchBot is OpenAI's. Claude-SearchBot is Anthropic's. PerplexityBot is Perplexity's. Amazonbot feeds Alexa and related products. If you want to be findable through an assistant, these are the ones that matter most.

Live fetch agents go to a page because a person asked about it right now. ChatGPT-User, Claude-User and Perplexity-User all behave this way. Someone pastes your link into a chat, the agent fetches it, and what it receives in that moment is what the person reads about you.

Training crawlers collect text to train models. GPTBot, ClaudeBot, Google-Extended, CCBot and Bytespider are in this group. Blocking them is a legitimate policy choice about your content. It is not a visibility decision.

Googlebot is its own case, and it is easy to get wrong. Google's AI Overviews and AI Mode are built from the ordinary Google index, which Googlebot builds. Google-Extended controls training, not Overviews. Blocking Google-Extended does not remove you from AI Overviews. Blocking Googlebot does.

Here is the consequence nobody expects. Blocking GPTBot does nothing to ChatGPT search, because ChatGPT search does not use GPTBot. A site can block every training crawler and stay perfectly visible in AI answers. A site can allow every training crawler and be invisible, because the search crawler is blocked or the page renders empty.

If your robots.txt has a rule that says "AI crawlers", no crawler reads it. They each look for their own token.

Three Layers That Decide Whether You Are Readable

When we audit a site for AI search readiness, we group everything into three layers, in the order the machine encounters them. If a layer below fails, nothing above it matters.

Layer one: Access. Can they fetch it?

This is robots.txt, and it is stricter than most people realise. A crawler looks for the User-agent group that best matches its own token, and only that group applies. If OAI-SearchBot has a group of its own, the star group is ignored for it entirely. If it has no group, the star group applies. Inside a group, the most specific path rule wins, and Allow beats Disallow at the same length. That is RFC 9309, and the large vendors follow it.

Two things follow from that rule. A blanket User-agent: * / Disallow: / blocks every crawler without a group of its own, which is usually all of the AI search crawlers. And a group written for one token of a vendor does nothing for that vendor's other tokens, so a rule for GPTBot leaves OAI-SearchBot untouched.

The most common way this breaks is not a decision at all. It is a robots.txt written for a preview or staging host, shipped to production unchanged. The rule that protected a work in progress is now hiding the live site.

A missing robots.txt, by the way, means everything is allowed. So does a group you never wrote. Absence is permission.

One more wrinkle worth knowing. Live fetch agents are less consistent about robots.txt than the index crawlers are. Perplexity documents that a user-initiated fetch happens regardless of robots rules, and others behave similarly. The practical meaning: a Disallow line can keep you out of an index, but it will not reliably stop a single fetch when somebody pastes your link into a chat. What that fetch receives still matters.

Layer two: Receipt. Do they get actual content?

This is where sites built with AI builders fail most often, and the failure is invisible from the browser.

A client rendered app ships an HTML file with one empty div and a script tag. Your browser runs the script and the page appears, looking perfect. A fetcher that reads the HTML and stops sees a title and nothing else. Google runs JavaScript, with delay and budget. The fetchers behind assistants mostly do not, and the ones that do are not the ones building the index.

You can test this yourself in one line, with no tools:

curl -sL https://yoursite.com | sed 's/<[^>]*>//g' | tr -s ' \n' ' \n' | head -40

Read what comes back. If the words you expect a customer to read are not there, an assistant is not reading them either. An allowed crawler that receives an empty page is worse than a blocked one, because it records that your site has nothing to say.

The fix is architectural, which is why honest tools describe it rather than promising to patch it: server side rendering, prerendering at build time, or at minimum a real body in the HTML for your main pages. Every major builder can produce a server rendered project.

A related failure looks different but has the same effect. A hosting challenge page, the "checking your browser" interstitial, is served to anything that cannot solve it. For a fetcher, a challenge and an empty shell are the same experience.

Layer three: Meaning. Can they tell what this is?

Once a machine has your text, it has to work out what the page is and who you are.

The cheapest win here is consistency of name. Your title, your Open Graph title and the name in your structured data should agree. "Acme", "Acme Widgets" and "acme.app" are three different entities to a parser. We check exactly this, and it disagrees more often than you would expect, usually because the builder filled one field and a human filled another.

Then structured data. Organization or SoftwareApplication markup on the home page with a name and a url. Article markup on articles. Valid JSON, because a block that fails to parse contributes nothing. FAQ markup only counts when the questions actually appear as visible text on the page, so markup copied from a template and never matched to the page is worse than none.

Finally, the opening paragraph. Assistants quote the first substantive sentence far more often than anything else on the page. If your first paragraph is a slogan, a machine has to guess what you do from your title. One plain sentence, "X is a Y for Z", near the top, is the highest leverage text on your site.

What About llms.txt?

llms.txt is a proposed file at the root of your site: a markdown map with your name, a one line summary and links to the pages that explain you. Coding agents and documentation tools read it.

Here is the honest position. It is not confirmed as an input to ChatGPT search, Google's AI features or Perplexity's ranking, and none of those vendors has said it is. Anyone selling you llms.txt as the thing that gets you cited is guessing.

It is still cheap and occasionally useful, so write one, keep the links live and on your own domain, and serve it as text rather than your HTML shell. Some hosts return the app for unknown paths with a 200 status, which makes a missing file look present. Then move on to the layers that actually decide whether you are readable.

The Questions Your Site Already Answers

There is one more angle that sits between "can they read you" and "will they mention you", and it is entirely in your control: which questions your pages answer in plain text.

Assistants answer from text that exists on a page. Someone asking what your product costs gets a real answer only if a page states the price as text. The same goes for who it is for, how it works, who makes it, whether it is secure, and what the alternatives are. These are the questions people put to assistants about any product, and they are answered from your site or from somebody else's.

Go through this list and check your own pages:

What is it? A plain sentence, near the top, not a slogan.

What does it cost? As text on a page, not inside an image or rendered only by script.

How does it work? Three to five steps under a heading that says so.

Who is it for? Named, in the description and in a heading.

Who makes it, and since when? An about page with real specifics.

How do I contact them? An address and a response time.

What happens to my data? A page that answers it directly.

Is there a free plan or trial? State it plainly, including when the answer is no.

How do I get started? A short guide, not "check the docs".

How does it compare to the alternatives? Your own page, or the competitor's page answers for you.

Every gap on that list is a question an assistant will answer about you from somewhere else.

How To Check Your Own Site In Ten Minutes

No tools required.

Fetch https://yoursite.com/robots.txt and read it. Look for a blanket Disallow, and for groups naming OAI-SearchBot, Claude-SearchBot, PerplexityBot or ChatGPT-User. Remember that a group for one token does not cover the others.

Run the curl command above on your home page and on two important pages. Count the words you get back.

View source and search for application/ld+json. Paste each block into a JSON validator. Check the name matches your title.

Read your first paragraph out loud. Does it say what the thing is, or does it sell a feeling?

Search your own pages for a price, a contact address and a sentence about data. If they are not there as text, they do not exist for a machine.

Fetch https://yoursite.com/llms.txt. If it returns HTML with a 200 status, your host is lying about it.

That is most of an AI readiness audit. It is unglamorous, and it is the part that actually moves.

What To Be Suspicious Of

The category is young and the claims are loud. Three specific ones deserve scepticism.

"Rank number one on ChatGPT." There is no ranking to buy or win. An assistant composes an answer from sources it retrieved. You can make your site retrievable and clear. Nobody can sell you a position.

"Our AI visibility score is 63." A single number across assistants, prompts and days, with no stated method, is a chart, not a measurement. Ask what it is made of. If the answer is a weighted blend of things nobody can reproduce, treat it as decoration.

"Add llms.txt and get cited." Covered above. It is a map file with no confirmed effect on the systems people care about.

The honest version of the pitch is smaller and more useful: here is what a named crawler is allowed to fetch, here is the text it receives when it does, here is what your markup says you are, and here is which of those a code change can fix.

Where BulkAudits Fits

We check readiness, and we repair what can be repaired safely. We evaluate every named crawler against your robots.txt with the same rules a crawler applies and quote the line that decided it. We fetch every sampled page without running scripts and show you the text a fetcher received. We parse your structured data, compare the names, and read your opening paragraph. Then the findings a deterministic code change can fix are opened as a pull request on your repository, which you review and merge. Nothing is written to your live branch.

We do not track prompts or citations, and we do not plan to. If you want share of voice reporting, use a tracker alongside; the two answer different questions and neither replaces the other.

AI Search Readiness checks whether AI systems can access and understand your website. It does not guarantee citations or recommendations. Anyone who tells you otherwise is selling a chart.

Start with the readiness checklist or read the six chapter guide. If you would rather have it done for you, audit your site and the three layers come back with the evidence behind each one.