Audit checks · blocked_for_ai_search
robots.txt blocks AI search bots from this page
robots.txt blocks AI search bots from this page: why it keeps you out of AI answers and how to fix it. Audit check blocked_for_ai_search (warning).
↓ Markdown for agents- Check id
blocked_for_ai_search- Severity
- warning
- Tool
site_audit
A path-level Disallow can hide your best pages (docs, pricing, comparisons) from AI search bots even when the rest of the site is open.
How to fix it
Unblock the path for AI search and user-fetch bots if it should be cited.
How LLMentions checks it
site_audit crawls your site the way AI bots do (raw HTML, no JavaScript), reads robots.txt for every known AI and search bot, and renders a few pages in a real browser for comparison. This check reports as blocked_for_ai_search with severity warning, which fails CI with --fail-on warning.
llmentions audit example.comThe result lists every issue with the pages affected, most severe first, and what changed since your last audit. See the site audit guide.