Wersja polska

How much AI bot traffic is really AI bots

The industry sells AI visibility reports built on a header nobody checks. That header is called User-Agent, and it is a claim rather than an identity — you can put anything in it. Below is a measurement from a live website: how many requests claiming to be an AI bot actually came from the network of the operator they named.

The number this is about

Among the requests that could be resolved, 71.2% were genuine. Counting everything, including what cannot be resolved at all — 20.4%. As of 21 October 2026, at 13:59 UTC.

We publish both numbers because the denominator changes the result drastically, and a page about measurement honesty cannot publish a figure the reader is unable to reconstruct. The first says: of the requests we know about, how many were real. The second: of all traffic, counting the unresolvable as unconfirmed.

“But I already have this in Cloudflare”

This is the most common reaction to this page, and it deserves to be taken seriously, because it is reasonable: hosting dashboards have shown AI bot traffic for a long time, so why count the same thing twice.

Because they count claims. On 3 September 2026 we checked what such a counter is worth. Two requests sent with an ordinary curl command from a Polish home connection, carrying a header claiming to be PerplexityBot, pushed the Cloudflare dashboard's “AI Answer retrievals” counter from 16 to 18, and Perplexity's own line from 5 to 7. Neither of those two requests came from Perplexity.

You do not have to take our word for it — this takes two minutes and can be repeated on your own domain. One command with a substituted header, and a look at the dashboard before and after.

This is not an accusation of dishonesty. Checking identity costs network lookups on every request, while accepting the claim is free — at the scale of a provider like that, the difference is real. The consequence, however, is that the counter measures what the bot said about itself, not who actually showed up. In our measurement the gap is 28.8% among the requests that could be resolved.

People also regularly suggest this is a configuration issue — that you have to “put the whole domain behind it”. That is not the point: this site is behind Cloudflare in full, and the dashboard still attributes requests to an operator based on the signature alone.

Check it on your own site

Do not take our word for it. The interactive version of this page has a generator that assembles the command for a domain you type in; here is the version with a placeholder address. It sends your own website two requests signed as PerplexityBot:

curl -s -o /dev/null -w "%{http_code}
" -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" https://your-domain.example/

First write down the current value of the AI traffic counter in your dashboard, run the command twice, wait a few minutes and refresh.

What you should see: nothing. Neither request came from Perplexity — they came from your own computer. What you will see: the counter two higher, attributed to Perplexity.

Only run this against a site you operate yourself. Two requests will not harm anyone, but sending forged signatures at other people's servers stops being a test and starts being a contribution to the problem.

Why we do not do it for you

Because we cannot check whether the domain someone typed in belongs to them. If that field sent requests from our server, anyone could enter someone else's address and forged traffic would land on a stranger's site — from our address.

This is the same problem the whole page is about. We cannot verify identity, so we do not act on a claim alone. Analytics dashboards face exactly the same question — and answer it differently.

“A plain log analyser used to be enough for this”

It was, and it still is — for counting. Tools that break traffic down by User-Agent have existed since the nineties and they do it well. The difference is not that we count, but that we check. A twenty-year-old analyser will show you GPTBot's visits; the table above shows how many of them passed an identity check. Those are two different numbers, and only one of them is fit to base decisions on.

What kind of measurement this is — scale up front, not in a footnote

This is one small website: a portal about home cheesemaking. The measurement has been running since 3 September 2026, that is 48 days, and covers 52,417 requests from 38 distinct bot identities across 387 networks.

This is not a representative sample of the internet and we do not pretend it is. The value of the data comes from something else: every request was checked by the same method, described below, and the result — including what we could not resolve — is published in full. The figures refresh themselves, so the longer the measurement runs, the stronger they get.

How you can impersonate a bot in the first place

Imagine posting a letter and writing someone else's name as the sender. The post office will deliver it — nobody checks whether the sender is who they say they are. The User-Agent header works exactly like that: a browser or a bot declares what it is, and the server accepts the declaration.

Writing a program that introduces itself as GPTBot takes one line. It requires no permissions and no expertise — you just type the text.

Verification means checking whether the letter really came from the stated address. Large operators publish lists of the network addresses their bots come from. If a request arrives from outside that list while carrying the operator's name, it is not that operator. That is the only way, and no declaration can get around it.

Analytics tools that show “AI bot traffic” mostly count declarations. Not because anyone is cheating — verification simply costs network lookups, and a declaration is free.

Who actually comes here

The more recognisable the bot, the more often someone impersonates it — because sites let known names through. The number of forgeries says something about the value of the brand, not about its honesty. A high figure in the “forged” column is not an accusation against the operator. Quite the opposite: it means the name is worth enough for a stranger to steal it.

The table lists only bots with at least three requests — at a single hit, a name in a ranking is noise rather than data.

The thing that stands out most

Impersonation happens to every recognisable name, search engines included — in this measurement, the number of requests claiming to be a search engine and exposed as forged is 844. What is not comparable is the proportions: for search engines the forged share is a minority, while for some AI bots there is not a single confirmed request. The current state is in the table above.

The reason is simple, and it is essentially the thesis of this entire page. Sites have been verifying Googlebot for twenty years: the checking mechanism is documented by Google, built into servers and widely used. Impersonating it gets caught — exactly like the requests above. Almost nobody verifies AI bots, so impersonating them simply works, and nobody counts it.

This is not a difference in technology. It is a difference in how many years a given name has been checked.

A crawler, or one specific person's question

Not all AI bots do the same thing, and most tools throw them into one bucket. The distinction matters in practice:

• ai_crawler — collects content for a model's future index. It moves on its own schedule.
• ai_user — a specific person asked a question in a chat, and the model went to this page to answer it. Behind every such request stands a living person and a concrete question.
• search_engine — the classic index. The boundary with AI no longer exists: Googlebot also feeds AI Overviews.
• seo_tool — commercial analytics crawlers.

The ai_user category accounts for 1,699 requests, of which 943 were forged. This is the group site owners care about most — and at the same time the one whose claims are trusted most readily, because it sounds like proof that somebody genuinely asked about us.

How to recognise an impostor without checking identity

Verification by network address requires lists that do not exist for every operator. But it turns out that the two groups behave differently enough to be separated without any list at all — which means the method will also work for operators that do not exist yet.

First signal: response size

Genuine bots receive 46,023 bytes on average, impostors 4,688. The reason is mundane: a real bot asks for a page that exists and receives its content. An impostor asks for files that are not there and receives a short error page.

Second signal: error rate

Genuine bots miss 2.3% of the time, impostors 71.1%. A real crawler follows the site map — so it knows what exists. An impostor guesses addresses from a ready-made list of popular configuration files, and most of its attempts hit nothing.

Two caveats, without which these numbers would mislead

First, this is an observation from a single case, not a law. Forged traffic in this measurement was dominated by one scanning run from one network. The signature is clear, but in practice it rests on a single actor — and should be treated that way until it repeats elsewhere.

Second, we cut this series in half ourselves. On 5 September 2026 we changed the server's behaviour: paths typical of scanners stopped receiving the application page with a 200 code and started receiving a real 404. That moves both indicators above — the error rate rises, the mean size falls — with no change whatsoever in bot behaviour. This is why the table has a “period” column and shows both halves separately instead of pretending continuity. A chart stitching them into one line would show a change that never happened.

What one group looks for, and what the other does

This is the sharpest distinction in the whole dataset. We do not publish the specific addresses that were requested — a published list of sensitive paths would be a ready-made map for the next scanner. The types are enough:

Confirmed bots asked for files containing code or secrets 0 times. Impostors — 2,140 times. With content it is the other way round: impostors reached for it 244 times out of 4,323 requests in total.

What “asset” means here: JavaScript bundles, stylesheets and images — the files every visitor's browser downloads. They are public by design and do not count as sensitive. With genuine bots they take up a fair share of traffic for a prosaic reason: every deployment changes their addresses, so the crawler comes back for a version that no longer exists.

The two groups' target sets barely overlap. These are not two kinds of bot doing the same job with different intensity. They are two entirely different activities carried out under the same names.

And this is where the real defence comes from, and it is not blocking: the scanners found nothing, because the files they asked for do not exist. There is no server-side language here, no admin panel at a well-known address and no database to dump. Blocking by bot name would be pointless — the name is false by definition. Blocking by network would also hit legitimate traffic, because large hosting providers host both.

How much of this traffic the site owner makes

For several days I checked whether robots can see this site's content — by impersonating ClaudeBot, GPTBot and PerplexityBot in turn. I left 3,062 visits which, in any ordinary analytics tool, would look like artificial intelligence taking an interest in my website. That is 5.5% of all robot traffic in the period.

I do not delete them. I subtract them. Every figure above is calculated after they are removed.

The rule is mechanical and stated in advance: a visit goes into the “owner's tests” category if it came from the network I work from, or carries an agreed marker in its signature — regardless of how that turns out for the statistics. The credibility comes from the rule being automatic and documented, not from my assurance that I am honest.

This entry is here for a specific reason. If someone testing their own site can produce 5.5% of its “AI traffic” in a few days, then every counter built on declarations is open to distortion even without ill intent.

Traffic that does not introduce itself at all

Everything above concerns requests that gave some name — because only such a declaration can be disbelieved. But part of the traffic gives none. The User-Agent field is sometimes empty, or holds a name that appears on no list: a script, a monitor, someone's tool written yesterday.

Since 7 September 2026 we count that traffic too. So far: 16,082 requests.

This figure has been measured for a shorter time than the rest of the page — since 7 September 2026, not since the start of the measurement. While the number is small, draw no conclusions from it: a zero at the start of a counter means the counter has just started. This page has already made that mistake once — see the paragraph below.

This number stands on its own and enters none of the percentages above. The headline figure on this page says how much traffic claiming to be an AI bot is genuine. Putting something into the denominator that never made such a claim would change what that figure measures — overnight, and without warning to the reader. So the percentages mean exactly what they meant yesterday.

Who they are, we will not find out. They left no name, and we deliberately do not record IP addresses. But what they are looking for we know precisely — and that is the more interesting information, because intent shows in the targets, not in the tool. A monitor asks for the home page and nothing else. A scanner asks for configuration files and backups. A reader asks for content. The breakdown by target is in the interactive “unsigned traffic” report on this page; sensitive paths appear there only as a count, because a published list of the addresses a scanner asked for is a ready-made map for the next one.

Do not compare this number with your hosting dashboard. Our counter sees only requests that reach our code. Files sitting ready on disk — images, stylesheets, scripts, the favicon — are served by the layer in front of it, and that traffic is not here. A hosting dashboard will count everything; we will count less. The difference does not mean either counter is wrong: they count different things. We checked this experimentally — a request for the site map reaches our code, a request for the favicon does not.

This is still not a counter of people. A browser sends headers that a simple HTTP client does not — request metadata, or at least a preferred language. A request carrying them does not enter this table at all. The boundary is arbitrary and can be wrong in both directions, but it is single, mechanical and stated in advance.

The reason we added this is instructive in itself. For the first days of the measurement, one line stood in the code: “if you do not recognise the name, record nothing”. Within a single day the same question came back from three different directions — the Cloudflare dashboard showed traffic our counter did not know about; visitor statistics turned up “users” with nothing behind them; and someone asked whether the site map had been read by something unrecognised. All three had the same cause. The counter was not showing zero because nothing was arriving — it was showing zero because it had no way to see.

Chronicle

Selected events worth remembering. No IP addresses and no paths.

10–13 September 2026 – the counter kept counting, the reports stood still

On the evening of 10 September we changed the rule that recognises our own test traffic: from a whole operator network to a marker in the request header. The change was meant to touch one rule, but it rewrote the entire function that describes every recorded visit, and it started from an older version of that function. One line fell out along the way: the one that gives each visit an address type, such as content, a technical file, or a file that scanners ask for.

Visits kept being recorded, just without that one field, and the database raised no error, because an empty field is allowed. The reports on pages read, missing content and scanner requests froze at 10 September, 18:52 UTC, and the incidents report stopped seeing new requests for sensitive files. We only noticed on 13 September, and by eye: the “last seen” column in all of those reports was stuck at the same hour. Nothing automatic caught it.

No data was lost, because the address type is not a measurement but a conclusion drawn from the stored address. It could be recomputed backwards without guessing, and the reports filled in for all three days. Had we stored only the conclusion and not what it is drawn from, those three days could not have been recovered.

13 September 2026, 15:39 – twenty-two well-known bot names from one network

From one network in the United States (AS396982) came a run of 205 requests that introduced itself under twenty-two different names, among them GPTBot, ClaudeBot, PerplexityBot, Googlebot, Meta-ExternalAgent and Amazonbot. In 80 cases verification contradicted the claimed identity; the remaining names belong to bots that cannot be verified. 82 requests concerned configuration files and source code, some of them aimed at the Vite development server, which many AI-built applications are made with.

They found nothing: none of those files exists on this site.

One of the names gives the forgery away by itself. Google-Extended is not a bot but a token used in robots.txt to opt out of Google using content to train its models. Google sends no requests under that name.

The forged names also included Meta-ExternalAgent and Amazonbot, which nobody had forged here before. This tool did not pick names; it worked through a whole list, so it forged every name roughly equally.

Our first reading was wrong. We took this run for a tool that checks whether a site lets AI bots in, because the incidents report showed zero requests for sensitive files next to it. It showed zero because of the failure in the previous entry. After the fix, the same run turned out to be a scanner.

7 September 2026 — a model described a file it had not fetched

We asked three models for verifiable facts from three of our files: how many addresses the sitemap holds and what the last one is, what the final section of llms.txt is, and what headline figure this page reports. The questions were phrased so the answers could be checked exactly — and so the counter would show whether the model actually came for those files.

One model answered all three with complete confidence, including a description of its method: “counting all the lines with addresses (…) gives 204 entries”. The counter holds no request for the sitemap during the test window; the only one that day came from somewhere else entirely, at a different time.

Two different things are worth separating here. The last address and the date in llms.txt are true but stale — they describe the state of a few days earlier. That is a remembered copy, and nothing is wrong with it except that it was presented as a fresh reading. The number 204 is something else: it was never true. No version of that file ever contained it. It was produced on the spot, together with a description of a counting operation nobody performed.

A fair caveat: we cannot rule out that the model fetched these files earlier, outside our measurement window — the pre-5-September data points that way, and that would explain the staleness. It does not explain a number that never existed, nor a sentence describing the counting as something just carried out.

For this page the first row of the table is the telling one. A page that measures how much “AI traffic” is genuine was described using a figure that did not come from it — and off by more than fourfold.

7 September 2026 — the model that admitted ignorance found our bug

In the same test another model replied: I could not fetch this file, and guessed it was a firewall on our side. The counter said otherwise: its requests arrived, both of them, to the second. The server returned the files correctly — the model's browsing tool could not handle a format other than HTML and reported its own limitation as somebody else's outage.

But the same model, asked about this page's content, replied that it had received “a table of contents resembling a sitemap, not the actual article”. That was true — and the fault was ours. This model does not identify itself by any name we knew, so it never reached the group served full content and got the fallback site map instead. Fixed the same day: whether a request comes from a human is now decided by the absence of headers a browser sends, not by whether a bot's name appears on our list. A list of names will always lag behind reality.

The pair is instructive on its own. The model that admitted ignorance gave us something worth fixing. The model that answered everything gave us nothing.

7 September 2026 — a third model said “I don't know”, and that was the best answer

A third model took the same test, also on a free tier. It came for all three files — we have its requests to the second. At the sitemap it hit the same limitation as the previous one: it could not read XML. And it did the one thing neither of the others did: it wrote that it would not give a number because it did not want to guess, and asked to be given the file's contents instead.

It read llms.txt correctly and currently, including the last-updated date — the same date the earlier model reported three months out.

For the figures on this page it gave values that are arithmetically consistent but do not match the counter at the moment of reading. We cannot settle this, and we record it as such: the page is cached for an hour and the numbers move quickly, so it may have received an older version — or reached into memory. Without a stored history of these figures we have no way to check, and guessing would be precisely what this page criticises in others.

What follows from this

Three models, one question, the same obstacle — and three different behaviours:

We repeated the test across five models and tiers, three of them paid. Combined result:

Three things here matter more than the ranking.

First: only one of six versions read the XML sitemap — giving the number of addresses, the last one and its date, all matching. Four admitted honestly that they could not, one made a number up. The practical lesson for anyone counting on a sitemap to “feed the models”: sometimes it does, more often it does not — and you cannot tell in advance which model it will be.

The first version of this paragraph read “none of the five read the XML sitemap”. It was true for an hour, until the sixth test. We leave the correction visible, because it is the same principle we apply to other people's figures: a conclusion drawn from a small sample goes stale faster than it looks.

Second: a higher reasoning mode did not produce a better factual answer. Version E reported this page's figures more precisely than anyone — and then, asked the simpler question of what the last section of a text file is, named a section that is not in it, justifying it with “the currently indexed version”. The simpler version D got it right.

Third, and this one is a practical warning: unable to read a public file, version E asked for access to a private code repository. The file sits at a public address and requires no permissions at all. The request was declined, rightly — it is an easy pattern to fall for: since I cannot read A, give me access to B. The distance between the two can be enormous.

The one model that refused to answer is also the only one whose other answers turned out to be true. That is not a coincidence or a neat ending — it is the same relationship this entire page is about. An answer is worth what the checking behind it is worth, not what its confidence sounds like.

All three tests were run on free tiers. We do not know whether paid versions would behave the same, and we make no claim about these companies — this is an observation from three conversations, not a measurement.

4 September 2026, 11:06 — six identities in nineteen seconds

From one network in the United States (AS1004) came a run of requests in which the same traffic introduced itself in turn as six different bots: CCBot, ChatGPT-User, ClaudeBot, GPTBot, OAI-SearchBot and PerplexityBot. The whole thing lasted nineteen seconds. The requests concerned configuration files, backups and source maps.

They found nothing — none of those files exists on this site.

A formal note that matters: we write that the requests came from the AS1004 network, not that “AS1004 is scanning”. A network operator rents out servers, and the actor is its customer, not the operator. The first phrasing is a verifiable observation; the second would be an accusation against a party we do not know.

5 September 2026 — no more answering “200 OK” to a question about somebody else's admin panel

Analysis of the run above revealed that some requests for non-existent files were getting a 200 code and the application page from us, instead of a 404. A scanner reads that as “there is something here” and comes back. It has been fixed — and along the way it turned out that this was deflating our own error rate, one of the very signals we use to recognise scanning. The server was corrupting a measurement it was itself feeding.

6 September 2026 — for three days we accused Amazon of what we were doing ourselves

Reverse DNS verification checks whether a hostname ends with the operator's domain. For Amazonbot we had crawl.amazon.com configured — a domain that does not exist. Amazon documents crawl.amazonbot.amazon. The consequence: every request from the real Amazonbot failed at step two and was recorded as impersonation. For three days this page showed Amazonbot as the most impersonated bot in the entire dataset.

The signal was in the data from the start: the reverse DNS method had a record of zero confirmations out of twenty-five checks. We noted that a day earlier and described the method as unproven — but stopped short of the conclusion that if it is unproven, then its “no” is not fit to publish either.

The verdicts could not be recomputed, because we deliberately do not store IP addresses — without the address there is no way to repeat the DNS query. So they were rolled back to “unverifiable”. This is the same principle we apply to other people's figures: a measurement known to have been faulty stops being evidence.

One fetch, many answers — the counter also undercounts

This whole page shows that AI traffic counters overstate, because they include impostors. Honesty requires the other side: in a different dimension they understate, probably by more.

On 7 September 2026 this page was fetched by one operator's bot at 09:49:29 — one request, one row in our counter. Then three separate conversations with that same operator quoted its content, two of them more than an hour later, without any new request reaching our server. We know because the counter recorded nothing further, while the answers carried figures frozen exactly at 09:49.

The effect is not limited to one operator. Two models from different companies, asked an hour apart, reported the same frozen state from a day earlier — with identical counts of forged and unverifiable requests. Both quoted a reading from the same moment; neither came for a new one.

One fetch served at least three answers. How many in truth we do not know and have no way to find out. Citation happens on the operator's side and leaves no trace with us.

So the number of requests in a log is not the number of citations, only a lower bound — and one of unknown tightness. Anyone counting “AI visits” to learn how often their content reached somebody's answer is measuring something other than they think, even after filtering out every impostor.

Something we do deliberately was confirmed along the way. Every figure on this page carries the date and time of the measurement, with a request to quote them together. The models did exactly that — reporting the number alongside “as of 7 September, 09:49 UTC”. A reader of such an answer knows they are seeing an hour-old snapshot rather than the current state. It is the cheapest protection against a quotation going stale that we know of, and the only one that works without the quoter's cooperation.

When a model cannot read something, it diagnoses your server

This is the most practical finding in the whole measurement, and the only one that can cost money.

On 7 September we published four tiny test files with the same content and different extensions — XML, plain text, JSON, and a copy of the sitemap. We wanted to learn which format models cannot read. The answer turned out to be unrelated to the question: format had nothing to do with it.

Throughout, the file answered anyone who asked: status 200, 246 bytes. We checked during and after. None of the guesses above — firewall, bot protection, missing file, serving method — had any basis in fact.

Three different architectures, three different causes of failure, all on the model's side. And three times the same reaction: describing someone else's infrastructure as broken, stated confidently and without the caveat that the problem might be their own.

Why this costs. Imagine asking an assistant why your site is invisible to AI and being told your server blocks bots. You start relaxing firewall rules, adding exceptions, asking your provider to change a configuration that was working correctly. You pay to fix something that was not broken — and weaken your defences along the way, guided by a diagnosis from a tool that never sent a request.

The question worth asking is simple and settles it in a minute: “is there any trace of your request in my logs?” If there is none, no hypothesis about the server has any basis, because the server never got the chance to do anything.

The answer came from the model that fetches. It read the same set of four files in full: XML, plain text, JSON and a copy of the sitemap under a different name — four keys, four hits, and four requests in our counter, all verified as genuinely coming from that operator. Format means nothing to anyone who actually goes out for the file. All that matters is whether they go.

This is worth pausing on, because the first impression runs opposite to the conclusion. A model that comes for the file leaves a trace: it gives a name, arrives from addresses its operator publishes, and can be checked, counted, or refused in robots.txt. A model reading somebody else's index is invisible to the site owner — absent from the logs, impossible to block, and quoting without anyone knowing.

It is an uncomfortable symmetry. The less a model troubles itself to reach the source, the less the source's owner knows they are being read — and the less control they have over it. This page counts those who come. About those who do not come and quote anyway, it knows nothing, and cannot.

One detail for the record: the code-running model had read two other files of ours correctly earlier the same day — its browsing tool works. This time it reached for the interpreter instead. It was not ability that failed, but the choice of tool — and the conclusion about somebody else's firewall was drawn just as confidently.

What we know about llms.txt — including how we skewed our own measurement

llms.txt is a proposed standard: a content map of a site prepared for language models. We have one and we are keeping it. But the data has to be reported plainly, including where it contradicts the reason for creating the file at all.

Ours was fetched four times. Three of those fetches we caused ourselves.

Testing whether models read the file, we asked three of them for its contents. Each genuinely went for it — and each of those requests landed in the counter as bot traffic. Our own-traffic filter does not catch this, because it recognises traffic by our network number and a marker in the signature, while the model arrives from its own infrastructure, under a real name, from an address that passes verification. To the counter it is an exemplary, authentic bot — because it is one. Only its cause was us.

Subtracting our own tests leaves one independent fetch. And even for that one we cannot currently say whether it passed identity verification, because our report does not split technical files into verified and unverified. That is a gap in the tool, not in the data — and it is fixable.

This trap is more insidious than the others because it works in one direction only: the more you test, the more your data confirms what you are testing. A dozen questions put to models about your own site is enough to make a small domain show “live AI traffic” that would not exist without them.

A stronger study says the same thing, with more confidence

Our sample is one site over a few days. Przemysław Charchan's study covers around 900 domains over 191 days: 1,227 requests for llms.txt files against nearly 45 million requests from AI bots in the same period. That is three thousandths of a percent. Among those asking for the file there was not a single verified AI bot — the traffic came from data aggregators, security scanners and people checking manually in a browser.

His sample is some forty thousand times larger than ours. When two measurements of such different scale agree, ours is not the evidence — it is at most an illustration, and that is how we treat it.

Why this happens — that part came from our own test

Numbers show the effect. We saw the mechanism during an unrelated check: we published four new files at public addresses and asked models for their contents. One sent no request at all and gave the reason outright — “cache miss, the search engine has no copy”.

A model built that way does not ignore llms.txt — it does not walk the files at all. It reads from a search index. A file prepared specially for it sits outside the path it travels, and no amount of improving its contents will change that.

What we do about it

We keep it. It costs nothing, and if the convention catches on we will have history from the day we started measuring. But we do not count on it, and we advise nobody else to. If you have one hour to spend: robots.txt is read here by more than a dozen different bots, and the home page is the second most fetched address on the whole site. Those are the doors people actually come through.

We are not the first — and that is good news

On 28 August 2026, a week before this counter started, Jakub Sawa published a study built on the same method: verifying bots by IP address rather than by name. He has been running it on jsseo.dev for eighty days and publishes both the methodology and the raw data.

He arrived at the same mechanism, independently and earlier: a pool of cheap servers dressing up as one bot after another (one address appeared under twelve names), scanning for configuration files instead of reading content, and edge infrastructure that verifies Google while letting the rest through.

But our numbers differ — and the difference is instructive. On his site 3% of GPTBot traffic is forged; in this measurement, 100%. That is not a contradiction, it is a different denominator. His domain is an SEO laboratory that genuine crawlers visit regularly, so impostors are diluted by authentic traffic. The real GPTBot did not visit a home-cheesemaking portal even once during the measurement window — so everything arriving under its name belonged to somebody else. Same method, two sites, opposite proportions, and both figures true.

This is also the strongest argument for having more such measurements. One measurement from one domain says something about that domain. Two independent ones, agreeing on the mechanism and diverging on scale, start to say something about the phenomenon — and show what that scale depends on.

How this is counted

Every bot request is recorded together with the outcome of an attempt to resolve its identity. The methods, in order of certainty:

Operator address list

The most reliable. OpenAI, Anthropic, Perplexity, Google and Microsoft publish files with the address ranges their bots come from. The check compares the request's address against that list. The result is decisive in both directions: an address on the list confirms, an address outside it carrying the operator's name means impersonation.

Operator network number

For operators that publish no list but do have a known network of their own. Weaker than an address list, because a network can be large, but still decisive for names nobody else has any business using.

Three-step reverse DNS (FCrDNS)

A classic method, used for years to verify Googlebot. It has three steps, and skipping any one of them makes it worthless:

1. ask for the domain name assigned to the address the request came from,
2. check that the name ends with the operator's domain — not “contains it”, because googlebot.com.example.org contains it and is not it,
3. ask for that name's address and check that it returns the same address you started from.

Without step three, anyone could simply set their own reverse record. And this is the method on which we made our own measurement error — described in the chronicle above. For the first days we were checking Amazonbot against a domain that does not exist, so every genuine request from that bot failed at step two and was recorded as impersonation. Fixed on 6 September 2026, and the earlier verdicts from this method were withdrawn to “unverifiable”.

Cryptographic signature (Web Bot Auth)

An emerging standard in which a bot signs its request with a key. Ultimately the best method available. For now we only record the presence of signature headers — we do not check their validity, so this is not verification and we do not count it as such. We record it to have historical data once the standard takes hold.

The table covers 52,200 requests — those where we recorded what settled the question. Earlier entries have that field empty not because verification failed, but because the column was added later than the counter.

What “unverifiable” means

That we cannot resolve it — not that the bot is suspect. Some operators publish neither address lists nor anything else that would confirm them. Putting those requests into “genuine” would inflate the result; putting them into “forged” would be an accusation without evidence. They stay separate.

This is the whole difference between this page and a dashboard that sells you a metric. Dashboards have no “don't know” category, because it looks bad. We have one, because without it the other numbers mean nothing.

Check us

What we ask of bots, and what we make available to them, is laid out in two text files: robots.txt (the rules we ask for) and llms.txt (a content map prepared for models). You can compare them against what is described above. They are raw configuration files, not articles — we point at them so that we can be audited.

What we deliberately do not record

We record no IP addresses and nothing that identifies a person. Recognising an impostor needs only the network number and the country — both describe a data centre, not a human being. We also do not record users' queries, because no bot passes them on and none will; that is somebody else's data.

The code is open

The counter behind the figures on this page is available in full: github.com/kitayMarek/ai-bot-verify (MIT licence). Verification code, database schema, public views and tests — everything needed to measure the same thing on your own site.

One file there matters more than the code: a list of the seven mistakes we made in the first week, each with the price we paid. A typo in a domain name that spent three days publicly accusing another company of impersonating itself. Files served without ever reaching the counter, producing a hard zero we read as “nobody fetches this”. A report filtering out exactly what we were looking for in it. Four of the seven produced results that looked entirely credible, and not one announced itself.

We publish it because building such a counter is easy; building one that does not lie is not. The code without that list would be a tool for producing more confident numbers, and there are enough of those already.

Data download

Every figure on this page is available machine-readable at bot-stats.json, under a CC BY 4.0 licence. One condition: credit the source.

The file contains aggregates only — the same ones shown above. It holds no IP addresses and no full paths.

Questions and answers

Is impersonating an AI bot illegal?

The User-Agent header itself is a declaration, and putting arbitrary text in it is not a crime. What it is used for is another matter: in the case described here it served to scan a server for passwords and keys, and that is something other than a choice of name.

Should I block AI bots?

Blocking by name does not work, because the name is precisely what an impostor forges — you will block the genuine one and let the fake one through. Blocking by network also hits legitimate traffic, because large hosting providers host both. The sensible order is the reverse: keep nothing on the server worth stealing, and measure who actually turns up.

Does robots.txt stop this?

robots.txt is a request, not a security control — it is honoured by those who choose to. One detail is worth remembering: a bot applies only the group of rules matching its own name, and rules from the general group do not add to it. A prohibition written only under User-agent: * will not cover a bot that has its own group.

Why is a cheesemaking portal measuring bot traffic?

Because it ran into it by accident. The counter was built to check whether language models see the site's content at all — and along the way it turned out that a substantial part of “AI traffic” is not that. The data is credible in one particular respect: we have no stake in it coming out one way or the other.

Will these numbers change?

Yes, and that is intended. The page is assembled live from the counter's current state, not from a frozen snapshot. That is why every figure carries a date — if you quote it, quote the date with it.

Related pages

• Wersja polska tej strony — the Polish original, with the interactive reports
• llms.txt — a content map of the site prepared for language models
• robots.txt — the rules we ask bots to follow
• Moja Serowarnia — why this site exists at all: home cheesemaking, in Polish