Key takeaways
llms.txt is a short Markdown index of your key pages; llms-full.txt is the full text of those pages in one file. Google says you can skip both, no major AI platform has said it uses them for citations, and 97% of llms.txt files in one large log study got zero requests. Indexing, crawler access and clear answers matter far more.
Every few years someone in our industry announces a file that will change how search engines see your website. Recently that file has been llms.txt, along with its heavier sibling, llms-full.txt. We have spent two decades in SEO, and every year of AI search since it arrived, and we have watched plenty of these files come and go. As a Toronto SEO firm working with Canadian and US small and mid-size businesses, we get asked about these two files almost weekly. Here is what each one does, what the evidence says as of October 2026, and how an owner should decide whether to bother.
Haven’t we seen this before? A short history of magic files
Search veterans get twitchy when a single file is sold as a shortcut. We remember clients paying to have fifty phrases stuffed into a meta keywords tag, long after Google said publicly that it ignores that tag for web ranking. We remember owners hand-tuning the priority value on every URL in their XML sitemap, a field Google has also said it ignores. We remember “submit your site to 500 search engines” packages, and authorship markup that put headshots in results until Google switched it off.
None of those ideas were foolish. Each one tried to hand search engines a cleaner signal. The catch was always the same: the search engine decided whether to listen, and usually it did not. That is the lens we bring here. llms.txt is not a scam. It is a proposal, and a proposal only matters once the platforms reading the web agree to use it.
Google sees the same parallel. John Mueller has compared llms.txt to the meta keywords tag, which is about as pointed as a comparison from Google gets.
What is llms.txt?
llms.txt is a plain Markdown file that lives at the root of your domain, at /llms.txt. Jeremy Howard of Answer.AI proposed it on September 3, 2024. The reasoning is sensible: language models struggle with menus, scripts, cookie banners and cluttered HTML, so give them a short, hand-picked map of your most useful pages in a format they can read easily.
The format is loose. Only one element is required: an H1 with the name of the site or project. After that you may add a blockquote with a brief summary, a few lines of notes, and H2 sections that list links with a short description each. A section titled “Optional” flags links a tool can drop if it runs short on space. The proposal also suggests offering Markdown (.md) copies of important pages.
What does an llms.txt file look like for a small business?
Below is one we wrote for a fictional commercial cleaning company in Hamilton that also serves western New York. Notice how little there is to it.
# Bayfront Commercial Cleaning
> Commercial and industrial cleaning for offices, warehouses and medical
> buildings in Hamilton, the GTA and western New York. Insured crews,
> night and weekend service.
Quotes are given after a site walk-through. We do not offer residential cleaning.
## Services
- [Office cleaning](https://www.example.com/office-cleaning/): nightly and weekly contracts
- [Warehouse cleaning](https://www.example.com/warehouse-cleaning/): floors, racking, high dusting
- [Medical office cleaning](https://www.example.com/medical-cleaning/): infection-control routines
## Company
- [About us](https://www.example.com/about/): ownership, insurance, service area
- [Contact](https://www.example.com/contact/): phone, email, quote request
## Optional
- [Blog](https://www.example.com/blog/): facility maintenance tipsThat is the entire file. If your site is already well organized, it takes about an hour to write, and it weighs less than a short email.
What is llms-full.txt, and how is it different?
Where llms.txt hands over a map, llms-full.txt hands over the territory. Instead of linking to your pages, it contains their full text, concatenated into one long Markdown document. The documentation platform Mintlify developed it together with its customer Anthropic, and it was later folded into the llms.txt proposal, as Mintlify explained in April 2025.
The size gap tells you who it was built for. A software company with hundreds of help articles benefits when a developer can paste the whole manual into a coding tool at once. A ten-page website for a renovation contractor gains very little, because its “full” file would simply be the homepage, services and about page in a different wrapper.
llms.txt vs llms-full.txt: the side-by-side view
| llms.txt | llms-full.txt | |
|---|---|---|
| What it holds | A curated list of links to your key pages, each with a one-line note | The complete text of your key pages, joined into one Markdown file |
| Typical size | Roughly 1 to 10 KB | Roughly 100 KB to several MB |
| Where it came from | Proposed by Jeremy Howard of Answer.AI, September 2024 | Built by Mintlify with its customer Anthropic, later folded into the llms.txt proposal |
| Designed for | Pointing a language model at your most useful pages | Letting a model or developer load an entire documentation set in one go |
| Effort for a typical business site | An hour or two | Setup time plus ongoing upkeep every time a page changes |
| Biggest risk | Drifting out of date | An out-of-date second copy of your whole website |
| Our call for a service business | Optional, low priority | Usually skip |
Put simply, one is an index and the other is a book. Most business websites are short enough that an AI crawler can read the real pages without help.
What does the evidence say about llms.txt in 2026?
We try hard to separate what the platforms say from what vendors sell. Here is where things stand.
Google’s position
Google has been steady on this. Its guide to optimizing for generative AI features in Search, covered by Search Engine Journal in May 2026, says showing up in AI Overviews and AI Mode is still SEO, and that no llms.txt or special AI text file is needed. In January 2026, when people noticed some Google properties hosting llms.txt files, Google said that was no endorsement. In June 2026 Google described any effect as “purely speculative”, with one practical caveat: if an AI platform that sends you business asks for the file, make one.
There is a wrinkle. Chrome’s Lighthouse 13.3, released in May 2026, added an experimental “Agentic Browsing” audit that checks for llms.txt. That audit concerns browser agents completing tasks on a page, not Search rankings, and Search Engine Journal noted that Google’s llms.txt guidance depends on which product you ask.
Adoption is climbing, requests are not
Originality.ai tracks more than 3 million sites, and PPC Land reported its numbers in July 2026: sites with llms.txt rose from 4,088 in June 2025 to 36,120 in May 2026, an 8.8x jump, while llms-full.txt went from 23 sites to 2,463. The same report carried the reality check. Ahrefs studied server logs across 137,000 domains in May 2026 and found that 97% of llms.txt files received zero requests. AI retrieval bots made up about 1.1% of requests to those files, and the most frequent named AI requester was GPTBot, which is OpenAI’s training crawler rather than the one behind ChatGPT search.
What the AI companies themselves say
OpenAI, Anthropic and Perplexity all publish llms.txt files for their own developer documentation. None of them has said publicly that its crawlers use llms.txt to pick sources for search answers or citations. Offering a file for your own docs and relying on everyone else’s for ranking are very different things.
Where llms.txt fits in a real SEO and AEO program
When we plan work for a business website, we rank tasks by how directly they affect whether ChatGPT, Google AI Overviews, Google AI Mode, Gemini, Perplexity and Microsoft Copilot can find, trust and quote the business. llms.txt sits near the bottom. In order:
- Get indexed in Google and Bing. AI Overviews, AI Mode and Gemini draw on Google’s index, and Copilot’s answers are grounded in Bing. An unindexed page is invisible to all of them.
- Let the right crawlers in. OpenAI’s crawler documentation says sites that block OAI-SearchBot won’t be shown in ChatGPT search answers. GPTBot, the training crawler, is a separate decision. PerplexityBot needs access for Perplexity to cite you.
- Answer real customer questions plainly. State the answer early, with specifics a competitor could not copy from you.
- Keep your business facts identical everywhere. Name, address, phone, services and service area should match across your site, Google Business Profile, Bing Places and directories.
- Earn mentions on sources AI tools already trust. Industry associations, local news, respected review sites.
- Then, if you like, add llms.txt. Cheap and harmless, with a low expected payoff.
Should your business publish llms.txt or llms-full.txt? A decision guide
You run a local or regional service business
Clinics, law offices, trades, accounting practices. Publish a short llms.txt if your developer can add it in an afternoon, and skip llms-full.txt. Your site is small enough for any crawler to read directly, and a giant text dump just creates a second copy you have to keep accurate.
You sell software or maintain real documentation
This is the case both files were built for. If customers or developers paste your docs into AI tools, produce both, ideally generated automatically by your documentation platform so they never fall behind.
You run an online store or a large content site
A curated llms.txt pointing at category hubs, shipping and returns policies and buying guides is reasonable. A full-text file of thousands of product pages is not; your product feed and structured data do that job better.
An AI platform that sends you customers asks for it
Then make it. That is Google’s advice, and ours.
A practical checklist if you decide to ship llms.txt
- Fix robots.txt and indexing first. The file cannot rescue a blocked or unindexed site.
- Use one H1 with your exact business name, matching your Google Business Profile.
- Write a two- or three-line blockquote: what you do, for whom, and where.
- List your 5 to 20 most important pages, each with a short factual note.
- Move the blog, archives and nice-to-have pages under “Optional”.
- Use full URLs that return a 200 status, not redirects.
- Upload it to yourdomain.com/llms.txt and confirm it opens in a browser as plain text.
- Review it whenever services, locations or contact details change.
- Check your server logs after a couple of months to see whether anything requested it.
Mistakes we see business owners make
- Paying a premium for it. If someone sells llms.txt as a standalone ranking service, walk away. It is a short text file.
- Saying things in the file that the website does not. Claims that live only in llms.txt create inconsistency, the opposite of what AI search rewards.
- Letting it go stale. A file listing a closed location or a dropped service is worse than having no file.
- Dumping a small site into llms-full.txt. Double the upkeep, no proven benefit.
- Fussing over llms.txt while robots.txt blocks AI crawlers. We still find business sites shutting out AI search bots by accident, usually through a security plugin or a copied template.
Our verdict
llms.txt is a reasonable, low-cost courtesy to AI tools, and for documentation-heavy companies llms-full.txt is a real convenience for developers. For a typical small or mid-size business website, neither file is a visibility lever today. Google says you can skip it, the major AI platforms have not said they rely on it, and the largest log study published so far shows almost nobody requesting it.
Our advice: publish a short, accurate llms.txt if it costs you an afternoon, skip llms-full.txt unless you have documentation, and spend your real effort on indexing, crawler access, clear answers and consistent facts. If you would like to know where your business actually appears in ChatGPT, Google’s AI features, Perplexity and Copilot, ask Avon for an AI visibility review. We will tell you what is worth fixing first, and whether llms.txt even makes the list.
Want a straight answer for your own website? Ask Avon Marketing & Media for an AI visibility review. We will tell you whether these files are worth your time, and what will actually move your visibility in Google and AI search.
Frequently asked questions
Will adding llms.txt help my website show up in ChatGPT?
There is no public evidence that it does. OpenAI publishes an llms.txt file for its own developer docs, but it has not said its search crawler uses other sites' files to choose sources. Allowing OAI-SearchBot in robots.txt and having clear, indexed pages matter far more.
Does Google use llms.txt for AI Overviews or AI Mode?
No. Google's guidance for its generative AI search features says you do not need llms.txt or any special AI text file. Pages need to be indexed and eligible to show a snippet, which is ordinary SEO work.
Can llms.txt hurt my SEO?
Not directly. It is a text file that search engines are free to ignore. The real risk is indirect: an outdated file that lists old services or wrong details can contradict your website, and consistency is something AI search tools do care about.
How much should I pay someone to create an llms.txt file?
For a typical business website it is a short task, often bundled into broader technical work rather than sold on its own. Be wary of anyone pricing it as a major project or promising rankings from it.
Do I need both llms.txt and llms-full.txt?
Most business websites do not. llms-full.txt suits companies with large documentation or knowledge bases. A small service site is better off with, at most, a short llms.txt and well-written pages.
Sources
- llms.txt proposal (llmstxt.org)
- Mintlify: What is llms.txt?
- Search Engine Journal: Google's new AI search guide calls AEO and GEO still SEO
- Search Engine Roundtable: Google does not endorse llms.txt
- Search Engine Journal: Google says llms.txt is purely speculative for now
- Search Engine Journal: Google's llms.txt guidance depends on which product you ask
- PPC Land: llms.txt adoption rises 8.8x but 97% of files get zero AI requests
- OpenAI: Overview of OpenAI crawlers
A Toronto SEO and Answer Engine Optimization firm helping businesses across Canada and the U.S. get found on Google and cited by AI since 2019.
About us →Get a free SEO and AI visibility audit.
Get My Free Audit
