llms.txt Generator

llms.txt is a proposed standard introduced by Jeremy Howard / Answer.AI in September 2024 at llmstxt.org. It's a markdown file served at the root of your domain (/llms.txt) that gives Large Language Models a curated navigation map of your most important content. Format: H1 site name, blockquote summary, then markdown sections with links. Unlike robots.txt (access control) and sitemap.xml (URL discovery), llms.txt is about content curation — telling LLMs which content matters most for understanding your site or product.

Not yet. It's a community proposal championed by Jeremy Howard (FastAI, Answer.AI). It's not an IETF or W3C standard. There is no formal compliance body. Adoption is partial as of mid-2026: Anthropic publishes their own llms.txt, several major dev-tool companies (Stripe, Cloudflare, Supabase, Cursor) publish theirs. OpenAI and Google have not formally committed to reading llms.txt files. Google's John Mueller has publicly questioned whether it adds value over existing standards.

Honest answer: partially and inconsistently. Anthropic referencing the format (and publishing their own) is the strongest adoption signal. Whether Claude, ChatGPT, or Perplexity actually fetch and prioritize content based on llms.txt during retrieval is not publicly documented by any vendor. Treat llms.txt as a low-cost optional signal — publishing one makes editorial intent explicit, costs near nothing to maintain, and may help adoption in the future even if today's vendors don't formally read it.

llms.txt is the short navigation map — H1 site name, blockquote summary, and curated section links pointing to canonical URLs. llms-full.txt is the expanded version with actual content concatenated — useful when you want an LLM to ingest your entire docs corpus in one fetch without crawling individual URLs. The pattern: publish llms.txt first as the index; add llms-full.txt later if you want LLMs to grab everything at once. Most major implementations (Anthropic, Stripe, Cursor) publish both.

Three files at your domain root, three different purposes. robots.txt is access control — tells crawlers what they may fetch. sitemap.xml is URL discovery — lists every page that should be indexed. llms.txt is content curation — tells LLMs which content matters most. They're complementary, not substitutes. Most well-configured 2026 sites have robots.txt + sitemap.xml as standard practice; adding llms.txt is an emerging additional signal specifically for AI crawlers.

Per the spec at llmstxt.org: H1 with site name → blockquote with short summary → optional H2 sections with bullet lists of important URLs in markdown link format. Example: # Acme Corp > Acme Corp builds developer tools for AI-native teams. We make X, Y, and Z. ## Docs - [Getting Started](https://acme.com/docs/start) - [API Reference](https://acme.com/docs/api) ## Blog - [Why we built Acme](https://acme.com/blog/why) Keep it short — under 200 lines is typical. The goal is curation, not exhaustive listing.

Be highly selective. The whole point of llms.txt is curation — telling LLMs which pages matter most, not duplicating sitemap.xml. List your cornerstone content: docs, key product pages, foundational blog posts. Don't list every blog post (that's what sitemap.xml is for). Most well-formed llms.txt files have 10-50 curated URLs grouped into 3-5 logical sections. If you find yourself listing 100+ URLs, you're probably duplicating your sitemap — step back and pick the 20-30 that genuinely represent your site.

At the root of your domain: https://yoursite.com/llms.txt — same location pattern as robots.txt. Serve it with Content-Type: text/markdown or text/plain and a 200 status. If you have a docs subdomain, you can also publish docs.yoursite.com/llms.txt. Some implementations (Anthropic) publish at the docs subdomain root rather than the main domain root.