Free Tool · Analyzer·Updated Aug 2026

AI Bot Log Analyzer

Paste your server access log and see which AI crawlers hit your site, what status codes they get, whether your citation bots are being blocked, and whether the requests are genuine, verified against published crawler IP ranges.

Paste your access log

Works with standard combined log format. It detects named AI user agents, tallies status codes, verifies source IPs against published crawler ranges to flag spoofed user agents, and flags 403/429s hitting your citation bots, the silent cause of missing AI citations.

AI bot traffic

Heuristic parse of user-agent and status fields. Citation bots blocked with 403/429 lose you visibility no llms.txt can recover, fix those first.

What it checks

Across one 515-million-event dataset, only 408 requests ever touched /llms.txt. The requests that matter, and that are often silently blocked, are the citation crawlers: OAI-SearchBot, PerplexityBot, Claude-SearchBot, Googlebot. This tool tallies your AI-bot traffic, shows the status codes each agent receives, and raises a flag when a citation bot is getting 403s or 429s.

It runs entirely in your browser. Paste a representative slice of your access log for the clearest read.

Does llms.txt do anything yet? →How AI crawlers index your site →

How to use it

  1. Paste an excerpt of your access log (Common or Combined format).
  2. See per-agent hit counts, status codes and llms.txt reads.
  3. Fix any 403 or 429 blocks hitting citation crawlers.

Frequently asked questions

Which AI crawlers does the log analyzer detect?

It flags the major citation and training crawlers, GPTBot, OAI-SearchBot, ChatGPT-User, PerplexityBot, ClaudeBot, Google-Extended and CCBot, and shows each one's hit count, status codes and whether it reached your llms.txt.

What should I do if bots are getting 403 or 429 responses?

A 403 or 429 to a named citation crawler means your firewall or rate limiter is silently blocking the bots that quote you. Verify the user-agent by reverse DNS, then allow-list the real crawlers in robots.txt and your WAF.

Is the AI Bot Log Analyzer free to use?

Yes. The AI Bot Log Analyzer is completely free, with no sign-up, no usage limits and no watermark on the output.

Is my data private?

Yes. The AI Bot Log Analyzer runs entirely in your browser. Nothing you paste or enter is uploaded, stored or sent to any server.

Related tools and reading

Keep going with the playbooks and tools behind this one.

How AI crawlers index your site →Robots.txt AI Generator →llms.txt Validator →