The AI + LLMO field guide.

What is LLMO

SEO optimizes for clicks off a results page. LLMO optimizes for citation inside an answer. Structured content, direct answers, and machine-readable indexes are the currency.

Crawlers that matter

AI crawlers split three ways: training (build model weights), retrieval (fetch pages to answer live queries), and user-fetch (a user asked the model to open a URL). Named bots: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended.

llms.txt

A plain-text index at /llms.txt that lists your best answer pages for AI systems. It goes at the site root, complements (does not replace) structured content on the page, and is increasingly the fastest path to being included in retrieval results.

Answer engines

ChatGPT, Claude, Perplexity, and Google AI Overviews pick sources they can quote cleanly, cite confidently, and attribute. Citable means: a direct answer near the top, a named source, and structured markup a model can trust.

Measurement

What to measure: crawler hits by class, citations against a target prompt set, and referral traffic from AI hosts. The meters on this site are the same ones CSToday runs on client sites.

Frequent questions

Do I need an llms.txt file?

It helps. An llms.txt file gives AI systems a machine-readable index of your best answer pages, which raises the odds that a retrieval crawler picks up the pages you want cited. It is not a substitute for structured content on the pages themselves.

Will blocking AI crawlers hurt my visibility?

Blocking training crawlers (GPTBot, Google-Extended) affects future model training, not today's answers. Blocking retrieval crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot) directly removes you from live answers. Blocking user-fetch agents (ChatGPT-User, Claude-User) removes you from user-initiated fetches. Treat the three classes separately.

How is LLMO different from SEO?

SEO optimizes for click-through from a ranked results page. LLMO optimizes for citation inside an AI-generated answer. The old signals still matter, but the winning surface is a well-structured answer block AI systems can quote verbatim.