Free tool
Can AI search engines read and cite your website?
AI assistants and answer engines (Perplexity, ChatGPT, Claude, Google AI Overviews) rely on robots.txt rules and llms.txt manifests to cite websites. If your server disallows their user-agents, your brand is invisible in AI answers. We inspect your live robots.txt and llms.txt files and help you generate standard configuration rules.
AI robots.txt and llms.txt generator
Decide which answer engines may read your site, and which may train on it. They are not the same choice.
What it actually does
Three steps, and none of them touch anything on the address you give it. Worth reading before you point a scanner at your own production site.
-
Step 1
Fetch robots.txt
One GET to /robots.txt to read your declared user-agent rules. No crawling and no form submission.
-
Step 2
Inspect AI crawler rules
We check permissions for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and 8 other major AI agents.
-
Step 3
Verify llms.txt
We check whether a standard /llms.txt markdown summary is published on your root domain.
Read-only. Nothing is signed in to, submitted or changed on the address you enter. See the methodology for how each result is graded.
The rest of it
This is one check of 48.
AI robots directives are one signal. The full scan also evaluates JSON-LD structured data, FAQ entities, Core Web Vitals, security headers, and domain authority.
Run the full free auditRun your first audit today
Start on the free plan, with enough searches to cover a city and enough audits to judge a shortlist. No card, and it does not expire.