Every serious decision a publisher makes about AI traffic starts with one question: who is actually reading this page? This week a small piece of research showed that the most widely copied answer to that question has a hole in it, and that the hole is written into Google's own documentation.
What did the research find?
On 5 November 2026, bot analytics company SenseGap published an analysis titled "Google's reverse DNS steps let any Google Cloud server pass as Googlebot". The finding is simple. Google's documented command-line method for verifying its crawlers has four steps: run a reverse DNS lookup on the visiting IP, check the hostname, run a forward DNS lookup, and confirm the IP matches.
Step two is the problem. Google's page tells site owners to "verify that the domain name is either googlebot.com, google.com, or googleusercontent.com". According to SenseGap, every Google Cloud virtual machine receives a reverse DNS name under bc.googleusercontent.com by default, and its forward lookup resolves back to the same IP. A script that follows the four steps as written will therefore accept a rented Google Cloud server as Googlebot.
SenseGap says it saw this in the wild. Across eight days of traffic (28 October to 5 November 2026) on four sites it operates, 89 requests to its own site claimed to be Googlebot, came from Google Cloud machines and passed the loose check (named-party figure, SenseGap). Those requests went looking for files such as .env, .git/config and cloud credentials.
Is this a Google bug or a reading problem?
Both, in a sense. The same Google page, last updated 20 March 2026, also carries a stricter table. It says genuine common crawlers like Googlebot resolve to hostnames ending in googlebot.com, special-case crawlers such as AdsBot to google.com, and user-triggered fetchers to gae.googleusercontent.com. Bare googleusercontent.com is not listed as a Googlebot hostname anywhere in that table.
So the precise rule exists. It is just not the rule in the four numbered steps, and the four numbered steps are what gets copied into firewall configs, WordPress plugins and log scripts. SenseGap reports no response from Google, and we have not seen one either.
Google also publishes IP range files for each crawler class (common-crawlers.json, special-crawlers.json, user-triggered-fetchers.json and others). SenseGap notes that checking a visitor against those files rejects the spoofing machines. That is the more reliable check, and it is the one fewer sites run.
Why does this matter beyond Googlebot?
Because the same research measured how often traffic claiming to be an AI agent is lying. SenseGap's share of fake requests, by claimed user agent, across its four sites (named-party figures, SenseGap, sampled Cloudflare analytics):
- CCBot: 100% fake (122 checkable requests)
- DuckAssistBot: 100% fake (115)
- Perplexity-User: 100% fake (98)
- PerplexityBot: 87% fake (153)
- GPTBot: 32% fake (291)
- ChatGPT-User: 19% fake (494)
- Amazonbot: 4% fake (2,968)
- OAI-SearchBot: 1% fake (11,672)
SenseGap says every fake AI bot visit across the four sites came from just seven Google Cloud IPs, and that about two thirds of fake requests were probing for secrets.
Treat the sample with care. Four sites, eight days, sampled analytics, and counts that SenseGap says shift by a few percent between runs. This is not an industry benchmark. But the pattern matches what larger studies already show. OPA Europe and Rossel found in January 2026 that 68% of media sites fail to detect basic bots and that 13% of AI bots ignore robots.txt (reported). Spoofed identity is not an edge case. It is the background condition.
What does it mean for publishers?
Start with measurement. If a share of what your logs call ChatGPT-User or Perplexity-User is a scraper wearing a costume, your count of real AI reads is wrong in both directions. You overstate agent demand where spoofers cluster, and you cannot see the genuine Live Search Agents that, by design, do not execute JavaScript and never trigger your analytics.
Then pricing. Every model for getting paid by AI, whether licensing, pay-per-crawl or monetising the read itself, depends on knowing which requests are real. A licence priced on inflated bot counts is a negotiating gift to the other side. A pay-per-crawl wall that a spoofed user agent can walk around is a toll booth with no barrier.
Then blocking. Publishers who block by user-agent string or by a loose reverse DNS rule are not blocking the bad actors. They are blocking the honest bots that announce themselves and letting the impostors through. That is the worst of both outcomes: content still gets consumed, and the publisher loses whatever leverage it had with the platforms that play by the rules.
Practical steps, in order:
- Audit any rule, plugin or script that accepts googleusercontent.com as proof of Googlebot. Require googlebot.com for Googlebot and gae.googleusercontent.com only for user-triggered fetchers.
- Verify against Google's published JSON IP ranges first, and use reverse DNS as a second check.
- Apply the same discipline to AI agents. OpenAI, Perplexity and others publish IP ranges for their bots; a claimed user agent from outside them should be treated as unverified.
- Report AI traffic as verified and unverified, separately. The second number is a security metric, not an audience metric.
What does it mean for advertisers?
Advertisers buying into AI answers, and brands measuring their visibility in LLM responses, should notice who sits upstream of that answer. The agent that fetched a publisher page to build a response is the moment of influence. If the industry cannot reliably tell a real ChatGPT-User fetch from a Google Cloud scraper, then any attempt to measure, verify or place a message at that moment needs identity checks built in from the start.
The same lesson is already familiar from programmatic display, where invalid traffic became a standing line item. AI answer surfaces will need their own version. Brands should ask any vendor selling AI visibility or placement how it verifies the agent on the other end, not just how it counts it.
What is the longer arc here?
The web's bot rules were built on good faith. Robots.txt is a request, not a lock. User-agent strings are self-declared. Reverse DNS verification was the first step towards proof, and this week shows how loosely even that step is commonly applied.
The direction of travel is towards cryptographic identity: signed requests, published key directories and standards such as Web Bot Auth. Until those are universal, the floor is IP-range verification done properly. Publishers who get identity right first will be the ones who can tell an AI platform, with evidence, exactly how many times it read them.
the blankspace read You cannot monetise a read you cannot verify. The whole argument for getting paid by AI rests on publishers knowing, request by request, which agent took what. Right now many of them are running a verification step that a rented cloud server can pass. Blocking on that basis is theatre and pricing on that basis is guesswork. Fix identity at the edge, then decide what each verified agent should be served. The read is the inventory, but only if you can prove whose eyes are on it.
Key figures
| Figure | Value | Provenance |
|---|---|---|
| Observation window | 28 October to 5 November 2026, four sites | Named-party figure (SenseGap) |
| Spoofed Googlebot requests passing loose check (sensegap.com) | 89 | Named-party figure (SenseGap) |
| Google Cloud IPs behind all fake AI bot visits | 7 | Named-party figure (SenseGap) |
| Share of fake requests probing for secrets | About two thirds | Named-party figure (SenseGap) |
| Fake share, CCBot / DuckAssistBot / Perplexity-User | 100% each | Named-party figure (SenseGap, sampled analytics) |
| Fake share, PerplexityBot | 87% | Named-party figure (SenseGap, sampled analytics) |
| Fake share, GPTBot | 32% | Named-party figure (SenseGap, sampled analytics) |
| Fake share, ChatGPT-User | 19% | Named-party figure (SenseGap, sampled analytics) |
| Fake share, OAI-SearchBot | 1% of 11,672 | Named-party figure (SenseGap, sampled analytics) |
| Hostnames accepted in Google's step 2 | googlebot.com, google.com, googleusercontent.com | Reported (Google Search Central documentation, updated 20 March 2026) |
| Media sites failing to detect basic bots | 68% | Reported (OPA Europe / Rossel, January 2026) |
| AI bots ignoring robots.txt | 13% | Reported (OPA Europe / Rossel, January 2026) |
Primary source, opened and verified
- SenseGap, "Google's reverse DNS steps let any Google Cloud server pass as Googlebot", 5 November 2026: https://sensegap.com/blog/googlebot-reverse-dns-google-cloud
- Google, "Verify requests from Google crawlers and fetchers": https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests