llms.txt: the complete guide

llms.txt is a markdown file at the root of your site that tells AI models what you do and where your best material lives. It takes twenty minutes to write. It is not an official standard, no major provider has committed to reading it, and it will not fix a site that is invisible for other reasons. It is still worth adding, and this guide explains exactly why and how.

llms.txt vs robots.txt

These two files get confused constantly. They do opposite jobs.

	robots.txt	llms.txt
Purpose	Restricts access	Explains content
Audience	Crawlers	Language models
Format	Directives (Allow, Disallow)	Markdown prose and links
Status	Universally honoured standard	Community proposal, growing adoption
If you omit it	Everything is crawlable by default	Models infer your site from the site itself

You need both, and they are not substitutes. A perfect llms.txt behind a robots.txt that blocks OAI-SearchBot is worth nothing.

The format

There is one hard rule: the file starts with a single H1 containing your site or company name. Everything after that is convention. A blockquote summary, then H2 sections grouping links, each link followed by a short description of what the reader will find.

# Acme Analytics

> Product analytics for B2B SaaS teams. Self-hosted or cloud,
> GDPR-compliant, with session replay and funnel analysis.

## Core documentation

- [Getting started](https://acme.com/docs/start): Install and send your first event in 10 minutes.
- [Event tracking](https://acme.com/docs/events): Schema design and naming conventions.
- [Self-hosting](https://acme.com/docs/self-host): Docker and Kubernetes deployment.

## Guides

- [Funnel analysis](https://acme.com/guides/funnels): Building and reading conversion funnels.
- [GDPR compliance](https://acme.com/guides/gdpr): Data residency and retention settings.

## Optional

- [Changelog](https://acme.com/changelog): Release history.
- [Engineering blog](https://acme.com/blog): Architecture write-ups.

The Optional section is a real convention, not filler. It signals material a model can skip when it is short on context budget. Use it for archives, changelogs and anything peripheral.

What makes a good one

• Describe, do not sell. A model parsing your file needs facts about scope, not adjectives. "Product analytics for B2B SaaS" beats "the world's most powerful analytics platform".
• Curate ruthlessly. Twenty well-chosen links beat two hundred. This is a map of what matters, not a sitemap.
• Write real descriptions. The one-line description after each link is doing most of the work. "Docker and Kubernetes deployment" tells a model something; "Learn more" does not.
• Use absolute URLs. Relative paths are ambiguous once the file is read out of context.
• Keep it current. A file pointing at dead URLs is worse than no file.

llms-full.txt

A companion convention: the same idea, but containing the full text of your key pages rather than links to them. It saves the model a round trip, which matters for documentation sites where the content is the product. For a marketing site it is usually overkill, and it creates a second copy of your content that will drift out of sync.

The honest assessment

Nobody can currently prove that llms.txt improves AI visibility, because no provider has confirmed reading it. Anyone telling you it is a ranking factor is guessing. What is true: it costs almost nothing, it forces a useful exercise in stating plainly what your site covers, and if adoption becomes real you are already there.

What actually decides whether an AI engine names you is more mundane: being indexed (especially in Bing, which ChatGPT searches through), letting the crawlers in, structuring answers so they can be extracted, and being described accurately on the third-party sources engines already trust. Our guide on how to rank in ChatGPT covers that groundwork. Add llms.txt after you have done it, not instead.