Free, no account, no cookies.

Which bots can reach your site?

Enter a domain. In under 30 seconds you will see what your site declares to crawlers today, and what changes for it on 15 September 2026.

No email required to see your report.

How it works

Enter a domain

No account, no verification — just type it in.

We read what's public

robots.txt, Content Signals, response headers — the same things a crawler sees.

See what changes on 15 September

Checked against Cloudflare's new crawler defaults, for your domain specifically.

Example report · example.com

Worth a look before 15 September.

This site is served through Cloudflare, appears to block AI training crawlers, and we found ads on 4 of the pages we checked. Those are the three things the 15 September defaults act on together.

What you declare today

robots.txt blocks 3 AI training crawler(s): GPTBot, ClaudeBot, CCBot.
Cloudflare is in front of this site (seen via response headers).

What changes on 15 September

From 15 September, Training and Agent are blocked by default on pages that display ads. We found 4 such page(s) here. Search stays allowed.
Search crawling is not blocked by the new defaults.

What we observed

The crawler checks below are indicative only. Cloudflare identifies crawlers by IP address and reverse DNS, not by the user-agent string, so our request claiming to be GPTBot is an unverified bot: it may be blocked where the real GPTBot is allowed, or allowed where the real one is blocked.

2 crawler user-agent(s) got a different response from an ordinary browser: GPTBot, PerplexityBot.indicative

What to decide before then

Look at the 4 ad-bearing page(s) listed below and decide whether the new Training and Agent defaults are what you want on them.
Open your Cloudflare zone's Security settings and look at the Search, Agent and Training controls. That page is where the 15 September behaviour applies.