How to Generate and Deploy an llms.txt File

llms.txt is a plain markdown file served at your domain root that tells language models which parts of your site matter. It is a convention, not a standard, and it takes about ten minutes to add. This guide covers the format, the deployment, and how to verify it is actually reachable.

What is llms.txt?

/llms.txt is a proposed convention for giving AI systems a curated, machine-readable summary of a website. Instead of asking a model to infer structure from HTML, it hands over a short markdown document with a project summary and a list of the most important links. The companion file /llms-full.txt carries the expanded content itself.

What goes inside the file?

The convention is deliberately simple: one H1 with your site name, a blockquote summary, then H2 sections of annotated links.

# Your Company

> One or two sentences describing what you do and who it is for.

## Core pages
- [Product overview](https://example.com/product): what it does.
- [Pricing](https://example.com/pricing): plans and limits.

## Documentation
- [Getting started](https://example.com/docs/start): setup in five minutes.

## Optional
- [Blog](https://example.com/blog): release notes and research.

Two rules matter more than the format: the summary line is what a model is most likely to quote, so write it as a fact-rich sentence rather than a slogan; and every link must resolve, because a file full of 404s is worse than no file at all.

Where does the file go?

It must be reachable at the exact path https://your-domain.com/llms.txt — the domain root, not a subdirectory. How you publish it depends on your stack: on Next.js, drop it in public/llms.txt; on most static hosts, put it in the publish directory. Avoid redirects: serve it directly with a text/plain content type.

Does llms.txt actually improve AI visibility?

Partly, and less than most vendors claim. Google has stated it does not use llms.txt in Search. Crawler support is inconsistent, and no major AI provider has committed to reading it as a ranking signal.

What it genuinely helps with today: documentation sites and coding agents that fetch context deliberately, and any retrieval system that accepts a curated entry point. The honest framing is one small, cheap signal — worth adding, not worth treating as a ranking factor.

How do I verify it is live?

1. Request https://your-domain.com/llms.txt in a browser. It should render as plain text, not download or 404.
2. Confirm the response is 200 and the content type is text/plain.
3. Check that robots.txt does not disallow the path, and that your firewall does not block crawler user-agents.
4. Click every link in the file once. Broken links are the most common defect.

You can run all of this automatically with the llms.txt Studio, or audit crawler access first with the free GEO scanner.