llms.txt: What It Is and How to Create the File for AI

Just as robots.txt tells search engines what to crawl, a new file has been proposed to guide artificial intelligence through a website’s content: llms.txt. At a time when tools like ChatGPT and Perplexity pull content from the web to build their answers, having a file that presents your site to these models in an organized way has become a hot topic for anyone working on digital presence.

This guide explains what llms.txt is, what it’s for, how to create the file, what to include and, importantly, what to realistically expect from this still-young proposal. The goal is to separate what the file actually does from the hype that sometimes surrounds it.

Quick answer: llms.txt is a text file proposed as a standard for presenting a website’s content to AI in an organized, easy-to-process way. It lives at the root of the site and lists the most important pages with links and short descriptions, in Markdown. The proposal was introduced in 2024 by Jeremy Howard of Answer.AI. Keep in mind that llms.txt is still a proposal, not a standard officially adopted by the major models, so its practical effect is promising but still limited.

What Is llms.txt?

llms.txt is a plain text file placed at the root of a website, designed to present that site’s content to AI clearly and in an organized way. The idea is to give language models a curated guide to the most relevant content on the site instead of leaving them to dig through everything on their own.

The inspiration comes from files that already exist to guide bots. robots.txt tells search engines what they may crawl, and the sitemap lists the site’s pages. llms.txt follows the same guiding logic, but for language models, the LLMs, which consume content differently from traditional search engines.

The proposal was introduced in September 2024 by Jeremy Howard of Answer.AI, and the specification is documented at llmstxt.org. That origin is worth remembering: it’s a community proposal, not a standard imposed by the companies building the major models.

What Is llms.txt For?

The problem llms.txt tries to solve is real. Modern web pages are full of elements a language model doesn’t care about: menus, ads, scripts, footers and interface code. When an AI tries to extract the useful content from a page, it has to separate what matters from the noise, and that doesn’t always work.

llms.txt offers a shortcut. Instead of the AI processing the whole page with all its noise, the file delivers a clean list of the site’s most important pages, with short descriptions and direct links, in a format the model reads easily. It’s like handing the model a curated table of contents instead of letting it flip through the entire site.

In theory, this helps AI better understand what your site is about, which content matters most and how the information is organized. For a company that wants to be understood correctly by AI, it’s an extra layer of guidance on top of the technical SEO work that already structures the site for search engines.

How to Create an llms.txt File

The structure of llms.txt is simple and uses Markdown, which is lightweight and easy to process. The file follows a basic layout.

It starts with the site name as the main heading, followed by a short sentence summarizing what the site is about. Next comes a brief description, in a paragraph or bullet points, giving context about the business or project. Then the most important pages are listed, grouped by section, each with its title, link and a short description of what the page contains.

The specification also includes an optional section for secondary content, which can be skipped when a model needs a leaner version. The idea is for the file to be curated, pointing to what really matters, rather than an exhaustive list of every page, which is the sitemap’s job.

Once created, the file is published at the root of the site, reachable by adding “llms.txt” after the domain, just like robots.txt. On WordPress sites, this can be done with a plugin or by uploading the file directly to the server.

What to Realistically Expect

Here’s the part many llms.txt guides leave out, and it matters so you don’t invest effort with the wrong expectations: llms.txt is still a proposal, not an established standard.

According to Search Engine Land’s coverage of llms.txt, it’s a proposed standard for AI content crawling, and adoption by the major models is neither guaranteed nor universal. In other words, creating the file doesn’t mean ChatGPT or Gemini will necessarily use it today. Adoption by the leading AI companies is still evolving.

That doesn’t make the file useless, but it calls for perspective. Creating an llms.txt is low cost and low risk, and it positions your site in case adoption grows, which is plausible given how much people are searching for the topic. What it isn’t: a magic fix that, on its own, gets your company cited by AI. The work that really moves the needle is still quality content, authority and a solid technical foundation, the combination that makes up GEO.

The balanced recommendation is to treat llms.txt as a low-effort complement within a broader AI optimization strategy, not as its centerpiece.

Frequently Asked Questions About llms.txt

Is llms.txt the same as robots.txt?

No, although the inspiration is similar. robots.txt tells search engines what they can and can’t crawl, with a focus on access control. llms.txt presents the site’s content to AI in an organized way, with a focus on helping models understand what’s relevant. They complement each other, each with its own purpose.

Do the major AI models already use llms.txt?

Adoption is still evolving and is neither guaranteed nor universal among the leading models. llms.txt is a recent proposal, from 2024, and its use by the big AI companies isn’t yet an established standard. So creating the file is a forward-looking positioning move, not a guarantee of immediate impact.

Is it worth creating an llms.txt anyway?

For most sites, yes, because the cost and risk are low. Creating the file takes little time and positions the site in case adoption of the standard grows. The key is realistic expectations: it’s a complement within a broader AI optimization strategy, not a standalone solution.

Where does the llms.txt file go?

At the root of the site, reachable by adding “llms.txt” after the domain, just like robots.txt. That standard location is what lets models that support the format find the file automatically.

Do I need to know how to code to create an llms.txt?

Not necessarily. The file uses Markdown, which is plain text with a light structure. Anyone can put together the basic structure, and on WordPress there are plugins that make creating and publishing it easier. The part that takes care is curation: choosing which pages to include and describing each one clearly.

Does llms.txt improve my Google SEO?

llms.txt is aimed at AI, not at traditional Google rankings. It doesn’t replace the sitemap or the SEO practices that affect search positions. Its role is on the AI optimization front, which complements but doesn’t replace SEO work for search engines.

Position Your Site for AI Search

llms.txt is a low-cost piece of a bigger movement: preparing your site to be understood and cited by AI. On its own it does little, but within a well-built strategy it adds up.

Malamute Digital handles AI search optimization end to end, from the technical foundation to content and authority. To build a strategy that prepares your site for both Google and AI, start with an SEO audit or explore our SEO consulting.