Key takeaways
The llms.txt file is a Markdown file placed at the root of a site to guide language models toward its useful content. As of September 2026, it brings no measured visibility gain. Google Search does not support it and AI engine crawlers almost never read it. It is mainly used by coding agents that read technical documentation.
- Google: “You don’t need to create new machine readable files, AI text files” to appear in search (guide updated on July 10, 2026).
- 97% of the 38,360 llms.txt files tracked by Ahrefs received no requests in May 2026 (Ahrefs, June 15, 2026).
- llms.txt does not block any crawler: to control access, use robots.txt and the CDN.
What is the llms.txt file?
The llms.txt file is a proposal by Jeremy Howard, published on September 3, 2024 on llmstxt.org. Version 2 of the specification dates from August 10, 2026. The idea: give language models a readable summary of the site, simpler to process than HTML pages loaded with navigation and scripts.
The specification requires only one element: the level 1 heading with the name of the site. It then provides for a summary in a blockquote, free-form paragraphs and lists of links under level 2 headings. An “Optional” section groups the links an agent can skip. Minimal example:
# Elev8 Lab
> Digital marketing blog: SEO, GEO, SEA.
## Guides
- [GEO guide](https://elev8-lab.fr/en/geo/): optimizing for AI engines
- [SEO guide](https://elev8-lab.fr/en/seo/): search engine optimization
## Optional
- [Glossary](https://elev8-lab.fr/en/glossary/)The same proposal suggests publishing a Markdown version of each page, at the same address followed by .md. Its authors mainly target use at answer time (inference), rather than model training.
llms.txt, robots.txt, sitemap: what is the difference?
Search results show frequent confusion between these three files. They have neither the same role nor the same status.
| File | Role | Status | Who reads it |
|---|---|---|---|
| robots.txt | Tell crawlers what they may crawl | IETF standard (RFC 9309, September 2022) | Search engines and declared crawlers |
| sitemap.xml | List the URLs to discover | Protocol recognized by search engines | Search engines |
| llms.txt | Summarize the site and point to its key content | Proposal, not standardized | Mainly coding agents |
The key point: llms.txt controls nothing. It does not block a crawler, does not allow any, and does not let you charge for access. Access control goes through robots.txt (see Google’s documentation on robots.txt) and the CDN settings. Our guide to the robots.txt file covers the useful rules.
Warning
Several articles present llms.txt as a way to “block” or “monetize” AI access. That is false. Charging for crawling exists, but it comes from a different system: Cloudflare’s pay per crawl, launched on July 1, 2025.
Does llms.txt improve visibility in AI?
No, according to all the public data available as of September 24, 2026. Three findings point the same way.
Google does not support it
Gary Illyes, from Google, stated in July 2025 that Google did not support llms.txt and had no plans to. John Mueller compared it to the meta keywords tag (remarks reported by Baseline Labs).
The official documentation points the same way: no AI text file is needed to appear in Google Search (Google Search Central, updated on July 10, 2026). And AI Overviews and AI Mode rely on the Google Search index.
AI crawlers barely read it
0.1%
Share of AI crawler visits that requested the llms.txt file in Otterly’s data: 84 requests out of 62,100 visits.
Otterly, data reported by Baseline Labs, 2026.
Ahrefs reaches the same result on a larger scale. Of 137,210 domains tracked in May 2026, 38,360 published a valid llms.txt. 97% of these files received no requests over the month (Ahrefs, June 15, 2026).
Among the few requests, SEO audit tools come first; AI assistants’ retrieval crawlers account for only 1.1%. Another finding: no AI crawler requested an llms.txt on a site that did not have one. AIs do not look for it on their own.
No measured effect on citations
No public study consulted isolates an effect of llms.txt on citations in ChatGPT, Perplexity or Gemini. The documented levers remain index crawler access, extractable pages and brand awareness. They are detailed in our article on ChatGPT SEO.
Who actually uses the llms.txt file?
The identified consumers are development agents: Cursor, GitHub Copilot, Claude Code, Cline or Aider. They read it when a developer asks them to look up documentation (Baseline Labs, 2026, source cited above). For a software or API vendor, an llms.txt that points to the documentation in Markdown helps these tools find the right page.
Even at Google, positions differ from one product to another. In late May 2026, the Chrome team added an llms.txt check to Lighthouse’s experimental agentic browsing audits (Ahrefs, June 15, 2026, source cited above). This check targets browsing agents, not ranking in Google Search.
It is a real use, but one that belongs to developer experience, not to search engine optimization. It is justified by the number of developers who integrate your product, not by a traffic goal.
Should you create an llms.txt file?
The answer depends on the type of site. It does not depend on a hope of visibility.
| Type of site | Recommendation | Why |
|---|---|---|
| Software vendor, API, technical documentation | Yes, if the documentation exists in Markdown | Coding agents use it to navigate |
| Editorial site, blog, media | Optional, low priority | No measured citation gain |
| E-commerce, brochure site, local service | Not a priority | The time is better spent elsewhere |
If you create one, keep three rules: a short file, links to pages that exist, and an update at every redesign. An llms.txt that points to dead URLs sends a bad signal to the agents that read it.
What has more effect than llms.txt
- Check access for AI index crawlers in robots.txt and at the CDN level. The details by vendor are on our GPTBot and AI crawlers page.
- Keep a clean sitemap, with indexable URLs only. See our XML sitemap guide.
- Make pages extractable: direct answer at the top, self-contained sections, sourced figures.
- Read the server logs to see which crawlers actually visit, with our log file analysis method.
To express a preference on how AIs use your content, another syntax exists in robots.txt. These are the content signals proposed by Cloudflare on September 24, 2025 (search, ai-input, ai-train). They are declarative preferences, with no technical enforcement (Cloudflare, Content Signals Policy).
So the llms.txt file remains a documentation tool, not a GEO lever (see our GEO guide). We will reassess it if a mainstream engine documents its use.
Frequently asked questions
Does Google use the llms.txt file?
Not for search. Google stated in July 2025 that it does not support it. Its documentation specifies that no AI-specific text file is needed to appear in search. AI Overviews and AI Mode rely on the Google Search index.
What is the difference between robots.txt and llms.txt?
robots.txt is a standard that tells crawlers what they may crawl. llms.txt is a proposed Markdown file that summarizes the site for language models. The first controls access, the second controls nothing.
How do you create an llms.txt file?
Create a text file named llms.txt at the root of the site. It starts with a level 1 heading with the name of the site and a short summary in a blockquote. Then come lists of links grouped under level 2 headings. Check that every link works.
Can the llms.txt file harm SEO?
No source documents a negative effect. The real risk lies elsewhere: spending time on it instead of checking crawler access, indexing and page quality. Those levers have a proven effect.
Sources
- Jeremy Howard, The /llms.txt file, v2, proposed in September 2024, modified on August 10, 2026. Accessed on September 24, 2026.
- Google Search Central, Optimizing your website for generative AI features on Google Search, updated on July 10, 2026. Accessed on September 26, 2026.
- Baseline Labs, Google says llms.txt does nothing for Search, 2026 (statements by Illyes and Mueller, Otterly data, uses by coding agents). Accessed on September 24, 2026.
- Ahrefs (Louise Linehan), We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read, published on June 15, 2026. Accessed on September 24, 2026.
- Cloudflare, Introducing pay per crawl, published on July 1, 2025. Accessed on September 24, 2026.
- Cloudflare, Content Signals Policy, published on September 24, 2025. Accessed on September 24, 2026.
- IETF, RFC 9309: Robots Exclusion Protocol, September 2022. Accessed on September 24, 2026.
- Google Search Central, Introduction to robots.txt. Accessed on September 26, 2026.
Digital marketing, SEO and GEO consultant
Article checked and updated by the author. Sources consulted on the date shown.
Cite this article
, . (2026, September 26). llms.txt: useful for GEO or not?. Elev8 Lab. https://elev8-lab.fr/en/geo/llms-txt/