Does llms.txt Work?

Short answer: It depends on what you want it to do. If you want to show up in ChatGPT, Perplexity or Google's AI answers, there's no good evidence that llms.txt helps. Google says you can ignore it for Search. A study of 137,000 sites found that 97% of llms.txt files got zero requests in a month. Where it can help is with AI agents, especially coding agents that read software docs.

Want to know if AI names you today? That's a better use of your time than a new file. MadeTheAnswer's free check asks ChatGPT, Perplexity, Gemini and Claude 5 buyer-style questions, one run each, and shows whether your product comes up. We work out your category and suggest likely competitors. Your score shows on screen right away, and the full report comes by email. No card. It's a quick snapshot, not a verdict.

What llms.txt is

llms.txt is a plain text file written in Markdown. You put it on your site, usually at yoursite.com/llms.txt. It gives AI tools a short summary of your site and a list of links to your most useful pages.

It was proposed in 2024 by Jeremy Howard, co-founder of Answer.AI and fast.ai. The proposal at llmstxt.org was updated to "v2" in August 2026. It describes the goal as giving "information to help agents use a website."

A basic file looks like this:

# Your Company

> One or two sentences on what you do and who it's for.

## Docs
- [Getting started](https://yoursite.com/docs/start.md): Set up in 5 minutes
- [Pricing](https://yoursite.com/pricing.md): Plans and what's included

## Optional
- [Company history](https://yoursite.com/about.md)

Only the top heading is required. The rest is a guide.

What it isn't

• It's not robots.txt. It doesn't block or allow anything. As the Ahrefs study puts it, "it controls nothing and blocks nothing."
• It's not a ranking signal that any major AI search tool has said it uses.
• It's not a sitemap. A sitemap lists every page for search engines. llms.txt is a short, hand-picked guide.

What the evidence says

Google: you can skip it for Search

Google's guide on AI features in Search lists tactics you can ignore. It says: "you can ignore tactics like 'chunking' content, creating unnecessary AI text files (like llms.txt), or pursuing inauthentic mentions."

According to the Ahrefs study, Google's John Mueller later said llms.txt is "not done for search." He called it a "temporary crutch, perhaps to save some tokens" for AI coding tools that read developer docs.

Ahrefs: almost nobody reads them

In June 2026, Ahrefs published a study of server logs from 137,210 websites, using May 2026 data. Its main findings:

• 28% of the sites had an llms.txt file. Ahrefs notes its users are more SEO-minded than most, so the real share across the web is likely lower.
• 97% of those files got zero requests that month. Not from bots, and not from people.
• Of the requests that did come in, 96% were from bots, but most of those weren't AI tools. They were SEO audit tools, tech scanners and other crawlers.
• AI bots of all kinds made 19.5% of the requests. AI search bots, like OAI-SearchBot and PerplexityBot, made just 1.1%.
• No AI bots asked for llms.txt files that didn't exist. In Ahrefs' words, "They never go looking."
• AI agents, like Anthropic's Claude Code, were the biggest AI group reading the files.

The study's verdict: "If your goal is showing up in ChatGPT, Perplexity, or AI Overviews, an llms.txt file is largely decoration."

The other side: agents and docs

llms.txt isn't useless. It just has a different job than many SEO posts claim.

The llmstxt.org page says the files are "used most heavily for software documentation, where coding agents follow them to find API references and tutorials." It also says "thousands of sites" publish one, and that Chrome's Lighthouse tool now checks for one as part of its agent browsing checks.

The big AI companies publish llms.txt files for their own developer docs. We checked on Oct 27, 2026, and both developers.openai.com/llms.txt and docs.anthropic.com/llms.txt load.

But even for docs, results vary. One developer on X wrote in June 2026 that the Astro web framework had removed its llms.txt files because "usage was less than 0.1% of their MCP server traffic." We haven't confirmed that ourselves, but it shows that even docs sites are asking if it's worth it.

So, should you add one?

If you do add one, do it safely

The Ahrefs study points out a real risk. AI agents are built to trust this file, and bad actors are already testing llms.txt for "prompt injection." That means hidden instructions meant to trick an AI. Ahrefs suggests treating the file like code:

• Keep it in version control and limit who can edit it.
• Get an alert when it changes.
• Keep it to plain links and short descriptions. Nothing that reads like an instruction.
• Only link to pages you control.
• Review anything your platform makes for you.

Also, link to the file from your site or docs. Ahrefs found that agents fetch llms.txt "when directed, not speculatively."

What helps more than llms.txt

If your real goal is to be named by AI tools, start here:

1. Don't block AI search crawlers. OpenAI says sites that block OAI-SearchBot "will not be shown in ChatGPT search answers." Check your robots.txt and your firewall or CDN.
2. Make your main pages clear. Say who your product is for, what it costs and how it compares. Put it in plain text on the page, not only in images.
3. Earn real mentions. AI answers often lean on reviews, lists and forum threads. See Why does ChatGPT recommend my competitor?
4. Follow a plan for each tool. See How to get your product recommended by ChatGPT and How to get recommended by Perplexity.
5. Measure before and after. Check whether AI tools name you now, then again after changes. See How to track AI visibility.

New to the topic? Start with What is generative engine optimization?

FAQ

Does ChatGPT read llms.txt? OpenAI hasn't said that ChatGPT search uses llms.txt to pick sources. In the Ahrefs study, OpenAI's search bot rarely fetched these files. OpenAI's training crawler, GPTBot, fetched them more often.

Does Google use llms.txt? Not for Search. Google says you can ignore "unnecessary AI text files (like llms.txt)" when it comes to its AI features in Search.

Will llms.txt hurt my site? Not on its own. The main risk is a file that's out of date or that someone changes to mislead AI agents.

Where do I put llms.txt? Usually at the root of your site, like yoursite.com/llms.txt. The v2 proposal also allows it in a folder, like /docs/llms.txt, to cover just those pages.

What's the difference between llms.txt and robots.txt? robots.txt tells crawlers what they may and may not crawl. llms.txt is just a guide to your content. It doesn't control anything.

Is llms.txt a standard? It's a proposal, not an official web standard. It's widely discussed, and some tools check for it, but no major AI search tool has said it relies on it.

Related guides

• What Is Generative Engine Optimization (GEO)? — Where llms.txt sits in the wider GEO conversation.
• AI Crawled My Site, but I Get No ChatGPT Traffic — Letting a bot in is not the same as being named in an answer.
• Does ChatGPT Use Google? — What is known about the search partners behind ChatGPT.
• How to Get Your Product Recommended by ChatGPT: A Step-by-Step Plan — The work that matters more than adding a new text file.

Check what AI actually says about you

A file won't tell you if you're being named. A test will. MadeTheAnswer's free check asks ChatGPT, Perplexity, Gemini and Claude 5 buyer-style questions, one run each, and shows you what they say. We work out your category and suggest likely competitors. Your score shows on screen right away, and the full report comes by email. No card. One run is a snapshot. The report shows whether AI names you, which competitors get mentioned, and which sites it cites, and each assistant's score links to the actual AI answers behind it. Paid weekly reports start at $29/month. They cover all 4 assistants, ask each question 3 times, and include a weekly Fix plan. See plans and prices. See a sample report.

---

Sources: llmstxt.org, "The /llms.txt file, v2"; Ahrefs, "We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read" (Jun 2026); Google Search Central, AI optimization guide; OpenAI, "Overview of OpenAI Crawlers"; OpenAI llms.txt and Anthropic llms.txt (checked Oct 27, 2026); @daviddalbusco on X (Jun 4, 2026).