Technical Layer · Auditor·Updated Aug 2026

Sitemap Discovery-Yield Auditor

Real-time AI indexers abandon a URL after three redirect hops. Paste your hop-sweep results to see how many of your declared URLs they can actually resolve, and which ones fail.

Paste your crawl results
One URL per line: the hop-sweep output, or url,status,hops CSV

A URL passes only if it returns 200 and resolves inside the three-hop ceiling. Real-time AI indexers abandon anything at three hops or more. Get the input by running the hop-sweep from the article against your own sitemap.

Discovery yield
0/ 1.0
Paste your crawl
Where the budget goes

The breakdown and worst offenders appear here.

What discovery yield measures

Discovery yield is the share of the URLs you declare that a real-time AI indexer can actually resolve into a citable document in one pass. A URL counts only if it returns 200 and lands inside the three-hop redirect ceiling, because indexers like OAI-SearchBot, Claude-SearchBot and PerplexityBot abandon a request at three hops with no error you can see. Most sites score between 0.60 and 0.85 on their first measurement, and the gap is almost always redirect chains and stale 404s, not missing pages.

How to use it

  1. Run the hop-sweep bash script from the article against your sitemap, or build a url,status,hops CSV yourself.
  2. Paste the lines here, one URL per line. Both the HOPS=.. STATUS=.. format and plain CSV are parsed.
  3. Read the yield: 0.98 and above is healthy, 0.90 to 0.98 is at risk, below 0.90 is failing.
  4. Fix the worst offenders top to bottom: collapse multi-hop chains to a single 301, and purge non-200 URLs from the file.

What is a good discovery yield?

Treat 0.98 as the target and 0.90 as the floor. Below 0.90, a tenth of what you declare is unreadable to a non-rendering indexer, which quietly caps how much of your site can ever be cited. Most sites start between 0.60 and 0.85 before anyone has looked.

Why does a URL fail at three hops?

Because real-time AI indexers enforce a hard ceiling of one to three redirects and drop the request after it, unlike Googlebot which follows around ten. Scheme, host and trailing-slash normalisation can burn the whole budget before your content is served, so the crawler never arrives.

What formats does the box accept?

Two. The output of the hop-sweep (lines like HOPS=2 STATUS=301 https://...), and plain CSV as url,status,hops. Any line it cannot parse is skipped, so you can paste a raw log and it will pick out what it can.

Does my data leave the browser?

No. Every line is parsed and scored entirely in your browser. Nothing is uploaded, stored or sent to any server.