# robots.txt is served

How the robots.txt is served check works, what a failure means, and how to fix it in the AI Crawler Access dimension.

- Dimension: AI Crawler Access (16% of the total score)
- This check: 2 points
- Check id: `robots-present`

## What does this check look at?

A request for /robots.txt returns HTTP 200 with a non-empty body. A 200 with an empty or whitespace-only body counts as not served, because it publishes no policy at all.

## What does a failure mean?

No readable robots.txt at the domain root.

## How do I fix it?

- Publish robots.txt. With no robots.txt every crawler is permitted by default, so this is a policy gap rather than a block.

## Why is it worth 2 points?

Highest weight of the twelve. A page that is blocked or unreadable has no path to being cited, so it dominates the total.

## What else is measured in this dimension?

- [No AI crawler is blocked](https://geo-scanner.ccie13192.com/checks/robots-ai-allowed/) — 6 points
- [robots.txt declares a sitemap](https://geo-scanner.ccie13192.com/checks/robots-sitemap/) — 1 point
- [Homepage returns HTTP 200](https://geo-scanner.ccie13192.com/checks/home-200/) — 2 points
- [Homepage is marked noindex](https://geo-scanner.ccie13192.com/checks/home-noindex/) — 2 points
- [sitemap.xml is present and well-formed](https://geo-scanner.ccie13192.com/checks/sitemap-valid/) — 3 points
- [robots.txt states a content policy](https://geo-scanner.ccie13192.com/checks/content-signal/) — 1 point

## How do I see my own result?

Run a free scan on your own domain at https://geo-scanner.ccie13192.com/ - the report lists every failed check with the evidence that produced it.

The full method is published at https://geo-scanner.ccie13192.com/methodology/