# A crawler policy is published

How the A crawler policy is published check works, what a failure means, and how to fix it in the AI Context Files dimension.

- Dimension: AI Context Files (5% of the total score)
- This check: 1 point
- Check id: `ai-context-robots`

## What does this check look at?

robots.txt is readable, so a crawler policy is published.

## How do I fix it?

- Publish robots.txt stating which crawlers are welcome.

## Why is it worth 1 points?

Deliberately low. Most tools in this category weight llms.txt heavily and imply it is a ranking factor; the evidence does not support that, so neither does this score.

## What else is measured in this dimension?

- [/llms.txt is present and structured](https://geo-scanner.ccie13192.com/checks/llms-txt/) — 4 points
- [A markdown alternate is declared](https://geo-scanner.ccie13192.com/checks/markdown-alternate/) — 1 point

## How do I see my own result?

Run a free scan on your own domain at https://geo-scanner.ccie13192.com/ - the report lists every failed check with the evidence that produced it.

The full method is published at https://geo-scanner.ccie13192.com/methodology/