Technical SEO for AI search: what controls your citations, and how to measure them

Most advice about "optimising for AI search" treats it as one thing you either do or do not do. In practice it is three separate things, controlled by different settings, and the same site can be excellent at one while being invisible in another. A page can be cited in Google's AI Overviews, unreachable to ChatGPT's search crawler, and quietly excluded from model training, all at the same time and all by design.

This article is about the technical layer underneath that: what actually decides whether an AI answer can cite your page, which controls you have over it, and how to measure what is happening, because most analytics setups cannot see it by default. If you want the robots.txt configuration itself, our earlier piece on AI crawlers covers it line by line. This one picks up where it leaves off.

Key takeaways

• Google asks for nothing new. To appear as a link in AI Overviews or AI Mode, a page must be indexed and eligible for a snippet. Google says there are no additional requirements or special optimisations.
• Your snippet controls now govern AI answers too. nosnippet, max-snippet and data-nosnippet limit what Google's AI features can use.
• Citation, retrieval and training are separate switches. Blocking a training crawler does not remove you from AI search, and allowing a search crawler does not opt you into training.
• Server-rendered HTML is a hard requirement for most AI crawlers. The major ones do not execute JavaScript.
• You can measure it, but not in one place. Search Console, the utm_source=chatgpt.com parameter and your server logs each see a different part of the picture.

What Google says it takes to be cited

Google's documentation on AI features and your website is short and more direct than most commentary about it. The key sentence: "There are no additional requirements to appear in AI Overviews or AI Mode, nor other special optimizations necessary."

The eligibility rule is equally plain. A page must be "indexed and eligible to be shown in Google Search with a snippet". If it can appear in a normal result with a text snippet, it can be a supporting link in an AI answer. If it cannot, nothing else you do will change that.

The same page explains why the fundamentals matter more rather than less. AI Mode and AI Overviews "may use a 'query fan-out' technique", issuing several related searches across subtopics and combining the results. A page that answers one subtopic cleanly and specifically can be pulled into an answer for a much broader question. The page that wins is often not the one targeting the headline query but the one that states a specific fact most clearly.

So the practical checklist for Google's AI features is the ordinary technical SEO checklist, done properly. The page has to be crawlable, indexable, fast enough to render, and specific enough in its text that a single passage stands on its own.

Your snippet controls now reach AI answers

This is the most under-used lever in the whole area. Google's robots meta tag documentation, updated in March 2026, states that nosnippet "will also prevent the content from being used as a direct input for AI Overviews and AI Mode", and that max-snippet limits "how much of the content may be used as a direct input" for the same features.

That gives you four levels of control:

The useful one for most businesses is data-nosnippet. It lets you keep a pricing table, a gated excerpt or a legal disclaimer out of AI-generated summaries while the rest of the page stays eligible. Blocking the whole page to protect one section is almost always the wrong trade.

Before you reach for any of them, weigh the cost. The Seer Interactive study below found that brands cited in AI Overviews earned 35% higher organic click-through rates than uncited competitors on the same queries. Seer is careful to note it cannot prove the citation causes the lift. But opting out is still a real decision with a likely cost, not a free privacy setting.

Citation, retrieval and training are three different switches

The most common mistake is treating "AI crawlers" as one category and blocking or allowing them together. Every major vendor now runs separate agents for separate purposes, and they document the difference.

Search crawlers decide whether you can be cited. OpenAI's publisher FAQ is explicit: "For your site content to be included in summaries and snippets in ChatGPT, make sure you aren't blocking OAI-SearchBot." Anthropic's Claude-SearchBot and Perplexity's PerplexityBot do the same job for their products. Google has no separate AI search crawler. Its AI features draw on the same index Googlebot builds.

User-triggered fetchers work differently. When someone asks ChatGPT, Claude or Perplexity a question and it opens your page to answer, that request comes from ChatGPT-User, Claude-User or Perplexity-User. OpenAI notes that for these, "robots.txt rules may not apply", and Perplexity says its user fetcher "generally ignores robots.txt rules", because a person asked for the page. Treat them like a browser, not a crawler.

Training crawlers and tokens decide whether your content trains models. GPTBot and ClaudeBot collect training data. Google-Extended is not a crawler at all, but a robots.txt token. Google's documentation states that it "does not impact a site's inclusion in Google Search" and is "not used as a ranking signal". Blocking it opts you out of Gemini training without touching your search visibility.

Our own robots.txt follows exactly this split. OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User and PerplexityBot are allowed. GPTBot, ClaudeBot, Google-Extended and Applebot-Extended are disallowed. Our content can be cited in AI answers but is opted out of training, and each of those decisions was made separately.

Server-rendered HTML is not optional

Googlebot renders JavaScript. Most AI crawlers do not. Vercel's analysis of its network, published in December 2024, found that "none of the major AI crawlers currently render JavaScript", naming OpenAI's three agents and Anthropic's ClaudeBot. They sometimes download JavaScript files, but they do not execute them.

The data is from 2024 and crawler behaviour can change. But no vendor has since documented JavaScript rendering for these agents, so the safe assumption is unchanged. If the text of your page only exists after client-side JavaScript runs, most AI systems see an empty shell.

In practice that means three checks:

1. Fetch your key pages with JavaScript disabled. Whatever is missing from that response is invisible to most AI crawlers.
2. Put facts in the HTML, not in widgets. Prices, specifications, opening hours and author details rendered by a client-side component are the first things to disappear.
3. Keep redirects and 404s tidy. A retired URL that still gets linked wastes the crawler's visit and gives it nothing to cite.

Structured data: consistency matters more than volume

Structured data does not get a page into an AI answer by itself. Google's own framing is modest. Its structured data policies say correct markup does not guarantee a rich result, and the AI features guidance lists "making sure your structured data matches the visible text on the page" as a best practice, not a lever.

The rule that matters is the matching rule: "Don't mark up content that is not visible to readers of the page." Markup that says one thing while the page says another is worse than no markup, because it gives any system reading both a reason to trust neither.

Where structured data earns its keep is consistency across templates. If your organisation's name, address, phone number and founding details appear in the same format on every location page, and your Organization markup uses one stable @id referenced from every page, an extraction system meets one version of the facts, not five. On our own site every article references its author and publisher by @id, so the byline, the profile page and the markup cannot drift apart.

How much traffic is at stake

The honest answer is less click-through per impression, but not zero, and the size of the effect depends on the query.

Pew's study tracked real browsing rather than surveys. About one in five searches in March 2025 produced an AI summary, and the share climbed with query length: 8% of one- or two-word searches, but 53% of searches of ten words or more. Users who saw a summary were also more likely to end their browsing session entirely: 26% of the time, against 16% without one.

Seer Interactive's click-through data, drawn from 25.1 million organic impressions across 42 organisations, shows the same direction from the publisher's side.

Two things are worth taking from that chart. Click-through fell on queries with no AI Overview too, so not all of the decline is the AI answer. And the steepest drop is on long, informational queries, which is exactly where a specific, well-structured page has the best chance of being the cited source. Google, for its part, says that clicks from pages with AI Overviews "are higher quality", with visitors more likely to spend time on the site. That is Google's claim, and the only way to test it for your site is to measure it.

Measuring AI visibility

No single tool shows all of this. Each part of the diagram above has its own instrument.

Google's AI features: Search Console. Google states that traffic from AI Overviews and AI Mode is "included in the overall search traffic in Search Console", within the Performance report's "Web" search type. It is not broken out separately, so watch trends on the long, question-shaped queries where AI answers appear most. Rising impressions with flat clicks on those queries is the typical signature.

ChatGPT: the UTM parameter. OpenAI's publisher FAQ confirms that "ChatGPT automatically includes the UTM parameter utm_source=chatgpt.com in referral URLs". In GA4 it shows up as a referral from chatgpt.com. Build a channel group or saved report that collects AI referrers, including chatgpt.com, perplexity.ai and claude.ai, so they are not buried in generic referral traffic.

Crawlers and fetchers: your server logs. Analytics tools run in the browser, and crawlers do not run your analytics script. The only record of OAI-SearchBot, Claude-SearchBot or ChatGPT-User visiting is in your server or CDN logs. Filter by user-agent and verify against the vendors' published IP ranges, since user-agent strings are easy to spoof. Log which paths they fetch, how often, and what status code they receive. A search crawler that repeatedly hits 404s or redirect chains is telling you where to tidy up.

Citations themselves: sample by hand. No tool yet reports reliably on every AI citation. A monthly sample, the same twenty questions your buyers ask, run through each assistant with the cited sources recorded, is crude, but it is honest about what it covers. Keep the question set fixed so the trend means something.

A checklist you can run this month

1. Confirm eligibility. Your key pages are indexed and show a normal snippet in Google Search.
2. Audit your robots.txt by purpose. Search crawlers allowed, training crawlers set deliberately, and nothing blocked by a blanket rule you did not intend.
3. Review snippet controls. Use data-nosnippet for the sections that should stay out of AI summaries rather than blocking whole pages.
4. Test without JavaScript. Every fact you want cited should be in the server-rendered HTML.
5. Check markup against the page. Structured data should describe only what a reader can see, with stable @id references across templates.
6. Keep pages fast. A slow first render is a slow page for every crawler. Our Core Web Vitals checklist covers the real thresholds.
7. Set up measurement. An AI referral channel in GA4, a log filter for AI user-agents, and a fixed monthly citation sample.

None of this replaces having something worth citing. AI systems choose sources that state specific facts clearly, and technical work only makes sure they can reach and read those facts. That is the part we handle in SEO engagements: the crawl, render and markup layer that decides whether good content is eligible at all. Our modern SEO guide covers the wider programme around it.