llms.txt: What It Is, a Real Example and Whether It Is Worth Doing

llms.txt is a plain Markdown file at the root of a website that gives AI assistants and coding agents a short, curated guide to the site: a name, a one-paragraph summary and a list of links to the pages that matter most. It was proposed in September 2024. Google says its Search ignores the file, and the crawler pages of OpenAI, Anthropic and Perplexity do not mention it. This guide shows a real file line by line, then gives a rule for deciding whether writing one is worth your time.

What is llms.txt and what does the format require?

The proposal lives at llmstxt.org. Its author, Jeremy Howard, published it on 3 September 2024, and the page was last modified on 10 August 2026. It describes the file as a way to give agents guidance for using a website, and it expects agents to read the file and then follow its links to the detail they need. The file is meant to stay small enough to fit in an assistant's context.

The format is Markdown with a fixed order, so programs can parse it. A file contains these parts, in this order:

1. An H1 with the name of the project or site. This is the only required part.
2. A blockquote with a short summary of the key information.
3. Optional paragraphs or lists, but no headings, with more detail on how to read the file.
4. Zero or more sections under H2 headings. Each holds a list of links in the form [name](url), optionally followed by a colon and a note.
5. An H2 section named "Optional", by convention for secondary links an agent can skip when it needs a shorter context.

The proposal says the file can sit at the root or at any path, and that it covers the pages under that path. It also says the file is used on demand, when an agent needs information while helping a person, and that its author expected it to matter for inference rather than for training.

What does a real llms.txt look like?

LiveMentions publishes its own file at livementions.com/llms.txt. It is 60 lines and a little over 4 KB. The image below shows its opening, one section and its closing section, with each part labeled by the rule it follows.

Read it in four parts.

• Line 1, the H1. The product name, as the proposal requires.
• Line 3, the blockquote. One sentence that says what the product is and what it does. It is the same description used across the site, so every page and the file tell an assistant the same thing.
• Line 5, the paragraph. A short paragraph of context that does not fit the summary. The proposal allows any text here except headings.
• Eight H2 sections with 31 links. Product, Platform, Solutions, Documentation and help, Free tools, Blog, Company and Optional. Most links carry a note after a colon that says what the page answers, which is what an agent needs to choose between two links. The Optional section holds only the two sitemaps.

Two choices are worth copying. The file lists the platform pages next to the documentation, so an assistant can find the definition of a feature and the page that explains how to read it. And it links an AI instructions page with the facts about the product and a preferred description, so an assistant that reads only this file still reaches the facts.

Who actually reads the file?

Here is what the primary sources say, as read on 15 October 2026.

A silence is not a denial. The three crawler pages are about robots.txt controls and user agents, so a missing mention proves nothing about what an assistant does at answer time. What the sources do show is that no provider publishes a promise that its assistant reads your file, and that Google says its search does not use it.

One more observation can be checked by anyone. Requests made on 15 October 2026 returned an llms.txt file from the developer documentation sites of OpenAI, Anthropic and Perplexity, and from the Gemini API documentation at a subpath. The files were about 6 KB, 85 KB, 37 KB and 37 KB. That shows these companies publish files for their own documentation, which is the use the proposal says is most common. It does not show that their assistants read files from other sites.

How does the file differ from robots.txt and sitemap.xml?

The three files sit side by side at the root of a site and do different jobs.

The proposal draws the same lines. A sitemap lists every indexable page, so it is usually too large for a context window and often lacks LLM-friendly versions of the pages. The new file is a short guided entry point. It can also complement robots.txt by explaining the content that robots.txt allows. It does not grant or deny access, so it never replaces a robots.txt rule.

Is it worth doing?

Judge it by cost, evidence and order of work.

The cost is small. A useful file has 10 to 30 links and a short summary, and an hour covers it. The maintenance cost is the real one: a file whose links point to retired pages teaches an assistant the wrong things, so it needs a review whenever your key pages change.

The evidence is thin for most sites. Google says its search ignores the file. No provider documents that an assistant reads it from your site. The proposal's own description says use is heaviest for software documentation, where coding agents follow it to find API references.

The order of work matters more. Three things come before it, because each one decides whether an assistant can use your pages at all:

1. Crawler access. A search bot that robots.txt or your firewall blocks cannot read any file you write. The Crawlability documentation shows how to read the status of each bot.
2. Content in the served HTML, not only after a script runs.
3. Pages worth citing, with a clear answer in the first paragraph.

A simple rule follows. Write the file now if you publish developer documentation or an API, because that is where the proposal says it is used. Write it after the three items above if you run a marketing site, treat it as low-cost housekeeping and never as a ranking tactic. Skip it, with no loss, if you cannot keep it current.

How do you create one in 30 minutes?

1. Write the summary. One or two sentences: what the product is, who it is for, what it does. Use the same description your site already uses.
2. Choose 10 to 30 pages. Pick the pages you would want an assistant to read first: product pages, documentation, pricing, contact. Group them under short H2 headings.
3. Add a note to each link. After a colon, say what the page answers in a few words.
4. Move the rest to Optional. Sitemaps, legal pages and long references go in the last section.
5. Publish it at /llms.txt. The proposal places the file at the root path or at a subpath that covers that part of the site.
6. Test it. The proposal suggests asking an assistant questions about your content, giving it only the file as a starting point. Wrong answers show which notes need rewriting.

Here is a skeleton to start from:

# Your Brand

> One sentence: what it is, who it is for, what it does.

Optional paragraph with context that does not fit the summary.

## Product

- [Product overview](https://example.com/product/): what the product does and who uses it
- [Pricing](https://example.com/pricing/): how plans and billing work

## Optional

- [Sitemap](https://example.com/sitemap.xml)

The proposal's integrations list says that several documentation platforms and CMS plugins can generate the file automatically. Review a generated file as carefully as a written one, because the notes decide whether it helps.

How does LiveMentions treat the file?

LiveMentions is an AI brand visibility platform. It tracks how often, and how favourably, a brand appears in answers from ChatGPT, Claude, Gemini and Perplexity, compares it with competitors, shows which sources the AI cites, and suggests ways to improve (the Optimize feature).

Optimize audits a site across seven areas, and Technical access is one of them. It checks robots.txt for blocked AI search bots, whether a sitemap exists, whether key content appears only after JavaScript runs, and whether the site has the file. Organization structured data is checked in a separate area; see schema markup for AI search for what is documented. The sample audit on the Optimize page marks a missing file as Optional, not as a failure. That matches the evidence above: useful housekeeping, never a ranking factor. The Optimize pillars guide lists what each area checks, and the help article on how findings are read explains the statuses.

Frequently asked questions

A five-minute decision

Open your site's root and check whether /llms.txt exists. If you publish documentation or an API, draft the file this week from the skeleton above. If you do not, check crawler access first, then content in the served HTML, then come back to it. Then use LiveMentions to see where your pages are cited across ChatGPT, Perplexity, Gemini and Claude, so you can judge any change by what the assistants say. The guide to what AI brand visibility is explains what to measure.