These two files send AI crawlers different but related signals: robots.txt controls which paths a crawler may fetch, while llms.txt curates the pages you want AI systems to use. If llms.txt promotes a URL that robots.txt blocks, crawlers get a contradictory message — you are advertising content you simultaneously forbid them to fetch. This checker surfaces those conflicts.

It flags pages or paths listed in llms.txt that fall under a Disallow rule in robots.txt for AI user-agents, AI bots that robots.txt blocks while llms.txt still invites consumption, and cases where one file exists but the other is missing or unreachable. The goal is a single, coherent stance toward AI crawlers across both files.

You enter a domain and the tool requests both /llms.txt and /robots.txt from that origin on our server, then parses each. robots.txt is parsed into user-agent groups with their Allow and Disallow rules; llms.txt is parsed for the page URLs and paths it references. It then cross-checks them and reports any mismatches it finds.

Decide your true intent. If you want AI systems to use that page, remove or narrow the robots.txt Disallow so the relevant AI user-agents can fetch it. If you do not want it used, remove it from llms.txt. Either way, make both files agree so crawlers receive one clear instruction.

Not necessarily. Many sites have robots.txt but no llms.txt, which is fine. The checker simply notes which files exist so you know your coverage. If you publish an llms.txt to guide AI systems, it is worth ensuring your robots.txt does not quietly block the very pages it highlights.

In practice robots.txt is the access-control file crawlers check first, so a Disallow there will stop a compliant bot regardless of what llms.txt says. llms.txt is guidance about which accessible content to prefer. That is exactly why a conflict matters — the robots.txt block wins, silently defeating the intent of your llms.txt entry.

No. The tool only fetches and parses two plain-text files and compares their rules with deterministic logic — no LLM or external AI API is involved. The files are read to perform the comparison and are not stored or used for training.