llms.txt is a Markdown file, served at the root of a website (/llms.txt), that gives AI language models a concise, structured summary of a site's purpose and most important pages — designed to be read directly by a model instead of crawled and parsed from HTML.
Where the spec came from
The llms.txt proposal was published in 2024 by Jeremy Howard (co-founder of Answer.AI and fast.ai). The core problem it addresses: HTML pages are built for browsers and human visual scanning — navigation bars, ads, cookie banners, JavaScript-rendered widgets — none of which is useful to a language model trying to quickly understand what a site is and where its key content lives. Large documentation sites are especially painful for a model's limited context window to crawl page by page.
llms.txt solves this with a plain-text, link-based table of contents a model can read in one pass. Adoption has grown quickly among developer-tool companies: Anthropic, Vercel, Cloudflare, Stripe, and others publish their own.
The format
An llms.txt file follows a specific Markdown structure:
# Site Name
> A one-sentence summary of what this site/product is.
Optional additional context paragraph — anything a model should
know before reading further.
## Section Name
- [Link title](https://example.com/page): optional short description
- [Another link](https://example.com/other-page)
## Another Section
- [Docs](https://example.com/docs): the full documentation
The required elements:
| Element | Rule |
|---|---|
| H1 | Exactly one, the site or project name |
| Blockquote | A single-sentence summary, directly after the H1 |
| H2 sections | One or more, grouping related links |
| Links | Markdown format - [title](url): description |
A real-world example
Here's a trimmed example in the style used by developer-tool companies:
# Acme API
> Acme is a REST API for sending transactional email.
## Documentation
- [Quickstart](https://acme.dev/docs/quickstart): get your first email sent in 5 minutes
- [API Reference](https://acme.dev/docs/reference): full endpoint documentation
- [Webhooks](https://acme.dev/docs/webhooks): event notification setup
## Guides
- [Deliverability best practices](https://acme.dev/guides/deliverability)
- [Migrating from SendGrid](https://acme.dev/guides/migrate-sendgrid)
Notice what's absent: no marketing copy, no pricing page, no blog archive. A good llms.txt is curated, not exhaustive — it should read like a table of contents a busy expert would hand a new colleague, not a full sitemap dump.
How to create one
- Write your one-sentence summary first. This is the single most important line in the file — it's what a model reads before anything else. Be concrete: "Acme is a REST API for sending transactional email" beats "Acme helps businesses communicate better."
- Pick 2-5 sections that group your most important pages — commonly Documentation, Guides, API Reference, or Product/Pricing for a marketing site.
- List only pages worth an AI assistant knowing about. Skip legal boilerplate, individual blog posts (link to a blog index instead, if at all), and anything duplicated elsewhere.
- Add a one-line description to each link when the title alone isn't self-explanatory.
- Publish it at your domain root:
https://yoursite.com/llms.txt, served as plain text, the same wayrobots.txtis.
Our llms.txt Generator builds this structure for you from a simple form, and can prefill link sections from your sitemap.xml.
Checking your work
Once published, run your file through our llms.txt Validator — it fetches the live file and checks the structural rules above (H1, blockquote position, H2 sections, link format) automatically.
Does it actually help?
Adoption among AI platforms reading llms.txt at query time is still uneven and evolving — it isn't yet a universal standard the way robots.txt is. But the cost of publishing one is close to zero: a single static file, no infrastructure changes, no downside. As more AI browsing tools and agent frameworks add llms.txt support, sites that already have a well-structured file are positioned to benefit immediately rather than scrambling to catch up.