Crawl-to-refer ratio: how much does an AI read per visitor?
An AI bot reads your pages. Sometimes it sends you a visitor. The ratio between the two can be counted, and it moves fast.
70,900:1
Pages read by Anthropic’s bot for a single visitor sent back. Cloudflare measurement, 19–26 June 2025.
Thirteen months later, the same vendor is reported at 2,237:1. Over the same window, Mistral is at 3,389:1 and DuckDuckGo at 2.5:1. SEOmator from Cloudflare Radar, 28 days to 21 July 2026.
Some read 3,389 pages to send you one visitor. Others read 2.
Every row carries its window and its source. None of these figures is ours.
| Operator | Pages read per 1 visitor | Window | Source |
|---|---|---|---|
| Mistral | 3,389:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| Anthropic (ClaudeBot) | 2,237:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| Perplexity (PerplexityBot) | 225:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| OpenAI (GPTBot) | 217:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| Microsoft (Copilot) | 35:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| Yandex | 26:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| Baidu | 12:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| ByteDance | 11:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| Google (Gemini / AI Overviews) | 4.6:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
| DuckDuckGo | 2.5:1 | rolling 28 days to 21 July 2026 | SEOmator, from Cloudflare Radar |
Checked on 1 September 2026 on SEOmator’s "GEO Data Report 2026" (Nick Sawinyh), a page last updated on 21 July 2026. The source it states is: Cloudflare Radar — Bot & Crawler Analytics, 28 days ending 21 July 2026. Cloudflare publishes the metric continuously on radar.cloudflare.com/ai-insights. These averages cover millions of sites on the Cloudflare network, never yours.
Starting point: 70,900:1 for Anthropic, 19 to 26 June 2025, measured by Cloudflare on its own network (article by David Belson and Sam Rhea, 1 July 2025). Thirteen months later, the SEOmator panel reports 2,237:1 for the same vendor, over a rolling 28 days. Two sources, two populations: read them side by side, never add them up — but the order of magnitude really has changed.
And how it moved through 2026.
| Operator | January 2026 | March 2026 | May 2026 | 1–21 July 2026 |
|---|---|---|---|---|
| Anthropic (ClaudeBot) | 56,969:1 | 11,870:1 | 11,934:1 | 2,363:1 |
| OpenAI (GPTBot) | 1,264:1 | 1,416:1 | 1,225:1 | 179:1 |
| Perplexity | 116:1 | 114:1 | 175:1 | 263:1 |
| Mistral | 22:1 | 39:1 | 40:1 | 6,021:1 |
| Google (Gemini / AI Overviews) | 5.1:1 | 5.5:1 | 5.0:1 | 4.0:1 |
SEOmator panel, checked on 1 September 2026 on a page last updated 21 July 2026. The July column is a PARTIAL month, 1 to 21: it is not comparable as-is to a full month. This 500+ site panel is not the same population as the Cloudflare network in the table above.
For scale: AI crawlers account for 20.3% of verified bot traffic, and AI search bots for another 6.5% — 26.7% in total. Cloudflare Radar, May 2026.
The number you read today will be wrong in six months.
Two orders of magnitude in seven months, both ways. Anthropic divided by 24, Mistral 274 times more, from January to July 2026. And 1,350 times apart between two bots on the same day.
The crawl-to-refer ratio is the number of pages an AI bot reads on a site for every human visitor the same platform sends back.
Cloudflare calculates it as follows: the number of requests from a platform’s user agents whose response is of type "Content-type: text/html", divided by the number of HTML requests whose "Referer" header contains a hostname belonging to that platform.
It falls, and hard
The SEOmator panel reports Anthropic at 56,969:1 in January 2026, then at 2,363:1 from 1 to 21 July 2026. Divided by 24 in seven months.
It rises just as hard
Over the same period and the same panel, Mistral goes from 22:1 to 6,021:1 — 274 times more. So there is no single trend to quote.
And it all depends on the bot
Over the 28 days to 21 July 2026, Mistral reads 3,389 pages per visitor sent back, DuckDuckGo 2.5. More than 1,350 times apart, on the same day.
No average, no snapshot, no figure read elsewhere says anything about your site. Your ratio has to be counted, not inferred.
Your own number, counted on your own server.
Two numbers, the same period, the same screen. And what it cannot do, spelled out.
The measurement happens server-side
AI bots do not run the JavaScript placed in your pages. Snorklee ships a server module for Node, PHP, Cloudflare, Netlify and Vercel, plus a WordPress plugin. Without it, those bots stay invisible.
Identity is verified where possible
A user agent can be faked. Three vendors publish the addresses of their bots: OpenAI, Anthropic and Perplexity. For those ten signatures, Snorklee checks the address against the official list. For the rest, it records what the bot declares, and says so.
Visitors sent back are counted separately
A visit arriving from ChatGPT, Perplexity or Claude is filed in its own channel, never under "Direct". That is the second number of the ratio, over the same time window as the first.
What Snorklee does not do
- Snorklee blocks nothing. It prepares a robots.txt block for you to paste on your own site; real blocking happens at your host or your CDN. A robots.txt is an instruction, not a lock.
- Snorklee counts no citations. It knows the site a visitor came from, never whether your page was quoted in an answer.
- Without the server module, AI bots do not appear at all: the script placed in the page measures humans only.
- A stealth bot presenting itself as an ordinary browser cannot be identified. Snorklee can describe a behavior, never name an actor that does not declare itself.
Once you know, you get to choose.
Three concrete decisions, and the measurement that judges them.
Write a robots.txt at the right level
GPTBot and ChatGPT-User are not the same bot. The first collects for training, the second reads a page while an AI answers someone. Blocking both under one rule cuts traffic you meant to keep.
Arbitrate bot by bot
A training bot that reads a lot and whose vendor sends you nobody is not a search bot that brings you visitors. Snorklee proposes restricting the first — and holds you back from blocking the second.
Measure the effect, 28 days later
A rule you put in place is judged on the same counter: did that bot’s hits go down? If the comparison window is missing, Snorklee says so instead of announcing a success.
Frequently asked questions about the crawl-to-refer ratio.
What is the crawl-to-refer ratio?
What is the ratio for Claude, ChatGPT and Perplexity?
Which AI bot reads the most for what it sends back?
Has Anthropic’s ratio come down?
Does the market average apply to my site?
How do I measure the ratio for my own site?
Can Snorklee block AI bots?
Does Snorklee know whether ChatGPT cited me?
How much does Snorklee cost?
Stop reading other people's numbers.
€8 per month ex. VAT: 200,000 pageviews, 10 sites, everything included. Hosted in Paris.