llms.txt is a markdown-flavored text file served at the root of a site (/llms.txt) that gives AI engines a hand-curated overview of what's there and which URLs are worth indexing.
It started as a proposal from Jeremy Howard in late 2024 and is now adopted by enough AI engines to be worth shipping. Think of it as the human-readable counterpart to robots.txt — robots says "don't crawl these," llms.txt says "here's the brand, here are the canonical pages."
What goes in it
The convention is markdown-style:
# Brand name
> One-paragraph summary of what the site is.
## Marketing
- [Homepage](https://...): one-line description
- [Pricing](https://...): one-line description
## Concepts
- [Term 1](https://...): one-line description
Short, scannable, no marketing fluff. AI engines that ingest the file get a concise map of the site's most-cite-worthy URLs without having to crawl every page.
Does it actually move the needle
It's not load-bearing — no AI engine has publicly committed to weighing llms.txt as a ranking signal. But several engineering teams have confirmed they consume it, and the cost to ship is zero. BCited's site readiness scanner checks for it as an AEO signal.