Skip to content
5.0 Upwork, 100+ reviews Adobe Certified since 2021 New: become your own Magento developer Learn more →
Hire me
Toggle Nav
My Cart
Magento glossary

What is Magento llms.txt?

Short answer Magento llms.txt means serving the llms.txt proposal (Jeremy Howard, Answer.AI, 2024) from your store, usually through an extension such as mage2kishan/module-llms-txt, which I wrote. The store answers at /llms.txt with a short markdown file: an H1 site title, a one-line blockquote summary, then an H2 section per content type with a bulleted link per page. AI tools such as Perplexity, Claude and ChatGPT browsing can read it as a map of the site instead of guessing from random pages. Think of it as a sitemap.xml written for language models instead of search crawlers.

Need llms.txt setup help? How it works

Written by Kishan Savaliya, Adobe Certified Magento & Hyvä developer, 10+ years on the platform

How it works
  1. Composer-install the llms.txt module 01
  2. Configure scope and per-type limits in admin 02
  3. Magento builds the file on request 03
  4. Serve the file at /llms.txt on the root domain 04
  5. Refresh on content change 05
Each step is explained below

Most common mistake Letting the file balloon past 50KB

How it works

Five steps from composer-install to a live /llms.txt file

llms.txt is a plain markdown file at the site root. The real work is choosing which pages go in, keeping the file short enough to read in one go, and keeping it current. This is how it works on a Magento store.

  1. 01

    Composer-install the llms.txt module

    Run composer require mage2kishan/module-llms-txt (or another llms.txt extension), then bin/magento module:enable Panth_LlmsTxt, setup:upgrade and, in production mode, setup:di:compile. You get a config section under Stores > Configuration > Panth > LLMs.txt and a storefront route that answers at /llms.txt. It runs on Magento 2.4.4 and later, and it works the same on Hyvä and Luma because the output is a text file with no theme involved.

  2. 02

    Configure scope and per-type limits in admin

    In Stores > Configuration > Panth > LLMs.txt you set how many entries of each type go in, for example Max CMS Pages (default 100). Use the priority URLs field to force in the pages that matter most (home page, main services, key landing pages), and keep out the noise: paginated category URLs, layered-navigation filter URLs, search results. I keep the file short on purpose, because a tool that reads only the first part of it should still see the important pages.

  3. 03

    Magento builds the file on request

    There is no file on disk to generate. When /llms.txt is requested, the module reads your CMS pages, categories, products and XML sitemaps, ranks the URLs, and writes each entry as a markdown bullet (- [Page title](https://example.com/page): description) under H2 sections per content type. The output starts with an H1 site title and a one-line blockquote, as the llms.txt proposal describes. The result is stored in its own Magento cache type, so later requests do not rebuild it.

  4. 04

    Serve the file at /llms.txt on the root domain

    A storefront controller answers at /llms.txt as text/plain; charset=utf-8. Check that nginx passes the path to Magento instead of looking for a static file (the module ships an nginx.conf.sample for this). The file has to load over HTTPS with no login, must not be blocked in robots.txt, and must not sit behind a Cloudflare bot challenge. AI crawlers that get a 401, 403 or 503 simply move on without telling you.

  5. 05

    Refresh on content change

    The cached output is cleared when you save content in the admin, so new pages show up on the next request. After a bulk import that skipped the admin, run bin/magento cache:flush. Then check with curl https://yoursite.com/llms.txt | head: the H1, the blockquote and the first H2 section should be right at the top. If a new page is missing, look at the per-type limits first. On my own site a full CMS limit was the reason new pages did not appear.

When to use

Four scenarios where shipping an llms.txt is the obvious next move

Adding llms.txt is a small job once the module is installed. These are the four kinds of store where I think it is worth doing soon.

  • Magento stores with rich editorial / glossary / blog content

    If the store has a real glossary, blog or help section, those are the pages AI tools like to quote when they answer a question. A curated llms.txt points them straight at those pages. Without it, they work from whatever they happened to crawl, which often means paginated archives and filter URLs instead of the articles you care about.

  • Sites optimising for AI Overviews, Perplexity, ChatGPT, Claude

    The llms.txt file is one of the few things at the site root written for AI tools rather than for search engines. It is still a proposal and no AI vendor promises to read it, so I treat it as a cheap extra, not a ranking factor. If you care about being cited in AI answers, a tidy llms.txt costs little and makes your best pages easy to find.

  • Service / agency / SaaS Magento sites with focused catalogues

    Magento isn’t only for big-catalogue retail, plenty of agency, SaaS, and service-business sites run on Magento with a tightly curated product list where each item is a discrete answer to a buyer query. For those sites, llms.txt is the perfect shape: 20-40 high-signal URLs, each with a one-line description that pre-summarises the value proposition for the LLM. Compare that to a 5000-URL sitemap.xml, the LLM has no way to know which 30 URLs actually matter without the curation layer that llms.txt provides.

  • Sites running a brand-mention strategy for AI training pipelines

    Brand-mention frequency in LLM training corpora correlates strongly with how often the LLM cites the brand in user answers. The fastest way to “introduce” a brand to the AI training pipelines without waiting for web-scale crawl coverage is a clean llms.txt file at the root domain, LLM crawlers prefer it because it lowers their parsing cost. The file is a self-contained, structured pitch for what the brand is and which pages summarise it best. Pair it with a focused content-marketing push and the citation rate climbs measurably within weeks.

Common mistakes

Three llms.txt mistakes that quietly kill the AI-citation lift

Most llms.txt failures aren’t catastrophic, they’re slow leaks. Audit your config and regeneration pipeline against these three before assuming the file is doing its job.

  • Letting the file balloon past 50KB

    LLM context windows are finite and most ingestion pipelines truncate long markdown contexts at a soft ceiling around 50KB. An llms.txt that crosses that line starts losing its tail-end entries, which is usually where the long-tail glossary pages and recent blog posts live. Tune max_cms / max_categories / max_products to keep the rendered file comfortably under that ceiling, prioritising the highest-intent landing pages. If you genuinely need the verbose, exhaustive variant, use the companion /llms-full.txt file for that, keep /llms.txt tight.

  • Including session-specific or paginated URLs

    Magento natively generates pagination and faceted-filter URLs (?p=2, ?___SID=U, faceted filter combinations like ?color=red&size=xl) that bloat sitemaps without adding signal. If those leak into llms.txt, the LLM’s context window fills with low-information URLs that all point to near-duplicate pages. Filter aggressively to canonical URLs only, the URL that rel=canonical points to, not the raw request URI. The Panth_LlmsTxt module pulls the canonical-tag URL by default; custom implementations need to do the same explicitly.

  • Forgetting to regenerate after content updates

    A stale llms.txt is worse than no llms.txt: it tells LLMs about pages that no longer exist or have moved, which signals dead links and erodes the trust the file is meant to build. Always tie regeneration to the standard content-save observers (cms_page_save_after, catalog_category_save_after, catalog_product_save_after) OR a daily cron entry, ideally both, with the observer doing fast async regeneration and cron acting as a nightly safety net. After every Magento content release, curl the file and eyeball it; it takes 30 seconds and catches every drift bug.

Related concepts

Five neighbour concepts most readers want to look at next. Click through for the full deep-dive.

FAQ

Magento llms.txt, frequently asked questions

  • Is llms.txt the same as robots.txt?
    No. The two files serve opposite purposes. robots.txt is an access-control file, it tells crawlers which paths are off-limits, and well-behaved crawlers respect those Disallow rules. llms.txt is a content-discovery file, it tells LLMs which pages are most worth reading and pre-summarises them in markdown so the LLM does not have to crawl, parse, and chunk the entire site to figure out what matters. Both live at the root domain, both are plain text, both are public, but robots.txt is gatekeeping (what NOT to fetch) while llms.txt is curation (what TO fetch, ranked).
  • Do LLMs actually read llms.txt yet, or is it speculative?
    Adoption is real and growing. Perplexity, Claude (when browsing is enabled), and ChatGPT’s browsing mode all check for /llms.txt when answering URL-grounded questions as of late 2024 / 2025, you can verify this by watching the network panel in Perplexity Pro during an answer involving your domain, or by checking your access logs for User-Agent strings like PerplexityBot or ClaudeBot fetching /llms.txt. Not every LLM does it yet, and not every browsing model does it on every query. The cost of shipping the file is near-zero (a single CLI command after installing the module), and the upside is meaningful and growing, this is the textbook case for a low-risk experiment.
  • Does Magento support llms.txt out of the box?
    No, Adobe has not shipped a built-in llms.txt module as of mid-2026, and there is no native llms.txt admin section in Commerce or Open Source. The two practical paths are: (1) install the mage2kishan/module-llms-txt extension used on this site, which provides an admin config section, a CLI generator, and a frontend route at /llms.txt; or (2) write a custom controller that builds the markdown from a hard-coded list of URLs. Option 1 is what 95% of sites should do, the extension handles the canonical URL filtering, observer-triggered regeneration, and per-type limits that a hand-rolled controller will get wrong on the first revision.
  • Should I list every product detail page in llms.txt?
    No, curate. List hero / flagship / lead products (the 20-40 SKUs that represent the brand and pull the most search demand), not the full catalogue. The whole point of llms.txt is to be the curated, signal-dense answer to “what does this site sell?” If you dump every SKU into the file, you defeat the purpose, you bloat the file past the 50KB ceiling, dilute the signal, and force the LLM to either truncate the tail or skip the file entirely. For the verbose, exhaustive variant, use the companion /llms-full.txt file. The Panth_LlmsTxt extension exposes max_products in admin precisely so you can pick the cap (somewhere between 20 and 80, depending on catalogue shape).
  • How is llms.txt different from a sitemap.xml?
    sitemap.xml is an exhaustive list of URLs intended for indexing search crawlers, it includes every canonical URL the site wants Google / Bing to know about, in machine-readable XML, with lastmod / changefreq / priority attributes that hint at crawl scheduling. llms.txt is a markdown-curated map of “what matters” with a human-written one-line description per entry. sitemap.xml is for indexing crawlers that will visit each URL; llms.txt is for LLMs that may only read the file itself, never visiting the individual URLs, the description on each line is what gets folded into the LLM’s answer. Different audience, different format, different curation logic, both useful, neither replaces the other.
  • Can I block specific pages from appearing in llms.txt?
    Yes. The mage2kishan/module-llms-txt extension has an exclusion admin config under Stores → Configuration → Panth → LLMS.txt → Exclusions where you can list URL patterns to suppress, useful for landing pages that are ad-campaign-specific, gated content, or deprecated URLs you have not yet 301’d. For hand-rolled implementations, simply do not include the URLs when generating the markdown. Note that excluding from llms.txt does not exclude from search crawlers, if you want to block from both, you also need a robots.txt Disallow and a noindex meta tag, since the three files target different audiences.
llms.txt audit

Want an llms.txt + AEO/GEO audit on your Magento store?

Send your storefront URL, I will check whether /llms.txt is served, validate the format against the v0.1 spec, audit the curation logic for noise, verify the regeneration hooks fire on content save, and confirm the file is reachable to Perplexity / Claude / ChatGPT crawlers. Written tuning plan, fixed-price quote, and earliest start date back to you in 24 business hours.

Get an llms.txt audit Hire me directly
Kishan Savaliya Online 10 AM to 7 PM IST Chat on WhatsApp