Two lines on a chart explain the 2026 web. One, automated traffic, keeps climbing: more than half of all requests now come from software, and the AI share of that is growing fastest. The other, the clicks AI systems send back to the pages they read, is close to flat. Every figure below measures one side of that gap, grouped by theme, with the original source named so each can be cited with confidence. Where a number comes from a different year or a secondary write-up, the text says so.
What share of web traffic is now bots?
Automated traffic has passed human traffic. Imperva's 2026 Bad Bot Report, published in April 2026, put automated traffic at more than 53 per cent of all web traffic in 2025, up from 51 per cent the year before, with human activity down to 47 per cent. Measured on requests for HTML content, Cloudflare Radar showed bots at 57.4 per cent of traffic and humans at 42.6 per cent over the seven days to 5 June 2026. The two measures differ because they count different things, but they agree on direction. HUMAN Security separately reported that AI-driven traffic grew 187 per cent across 2025.
How fast is AI bot traffic growing?
Among AI harvesters specifically, growth is steep. 51Degrees, a UK analytics company, examined more than three billion website visits in the year to 29 May 2026 and found AI bot traffic rising from 8 to 13 per cent of web sessions, an increase of 62.5 per cent. Amazon's bot accounted for about 4 per cent of all visits and Meta's external agent for about 3 per cent, and 51Degrees found the top four bots made up 85 per cent of sessions from the 72 AI-related bots it identified. Anthropic, OpenAI and Perplexity were each well under 1 per cent of visits in that dataset, a reminder that the loudest names in AI are not always the biggest crawlers on a given site.
How much of AI crawling is for training versus search?
Most AI crawling is the kind that cannot send a reader back. Cloudflare Radar's June 2026 snapshot showed training-related crawlers at 50.6 per cent of traffic to HTML content, while search-related crawlers were 10.7 per cent. TollBit's data points to a mix that is shifting rather than fixed: between the second and fourth quarters of 2025 it recorded training scrapes down 15 per cent, retrieval-augmented generation (RAG) bot traffic up 33 per cent and AI search indexer traffic up 59 per cent. Akamai's April 2026 State of the Internet research found 63 per cent of AI bot activity was from training crawlers and 25 per cent from fetchers, the bots that retrieve a page for a live user request.
How many AI bot visits are there for every human visit?
TollBit measured one AI bot visit for every 200 human visits in the first quarter of 2025 and one for every 31 by the fourth quarter, as reported by The Register in February 2026, while human visits to the sites it tracks fell 5 per cent over the second half of 2025. TollBit's Olivia Joslin said it could be this year that AI visitors become the dominant visitors to publisher sites. The ratio matters commercially because a bot visit that runs no JavaScript triggers no ad auction, no affiliate tracking and no analytics event.
How fast is agentic AI traffic growing?
Agentic traffic, meaning AI acting for an individual user, is the newest category. HUMAN Security's State of Agentic Traffic report for July 2026, published on 6 August, found media and publishers had the largest share of agentic traffic by sector at 43.5 per cent, overtaking e-commerce at 42 per cent for the first time. Discovery and research routes drew 76 per cent of agentic activity, while checkout and payment pages drew 2.6 per cent. In other words, agents are mostly reading and researching, not buying, and publishers are where much of that reading happens.
Who does the AI traffic come from?
Concentration varies by dataset, so treat any single league table with care. 51Degrees found Amazon and Meta together made up about 63 per cent of AI bot sessions in its sample. Akamai found 40 per cent of OpenAI bot requests targeted publishing organisations. TollBit's data showed OpenAI's ChatGPT-User bot averaged around five times as many scrapes per page as Meta, the next highest. Traffic that can be attributed to a named operator is far more useful to a publisher than a raw bot count, which is why identification comes before any policy decision.
What is the crawl-to-refer ratio, and why does it matter?
The crawl-to-refer ratio is the number of pages an AI company's bots fetch for each referral visit they send back. Cloudflare Radar tracks it per operator, using UTM parameters to measure referrals. The most widely quoted figures are from 2025. For July 2025, secondary analysis of Cloudflare Radar data put Anthropic's ClaudeBot at about 38,065 to one, OpenAI's GPTBot at roughly 887 to one and Perplexity at about 195 to one. Cloudflare itself cited 2025 ratios of 14 to one for Google, 1,700 to one for OpenAI and 73,000 to one for Anthropic when explaining its September 2026 crawler policy, and the specific number depends on the month and the bot measured. All of these sit against a search-engine ratio that is orders of magnitude lower. The pattern is the finding: content is read at industrial scale and very few readers come back.
How far have referrals and search traffic fallen?
Chartbeat, which aggregates data from thousands of publisher sites, reported in March 2026 that Google Search referrals fell 34 per cent and Google Discover 15 per cent between December 2024 and December 2025, and that average weekly page views across publishers fell 6 per cent. The fall was steepest at small publishers, down 60 per cent over two years, against 47 per cent for medium and 22 per cent for large publishers. Ahrefs found in February 2026 that AI Overviews were associated with a 58 per cent lower click-through rate for the top-ranking result, up from 34.5 per cent when it measured in April 2025. TollBit recorded click-through from AI applications falling from 0.8 per cent in the second quarter of 2025 to 0.27 per cent in the fourth.
Do AI referrals fill the gap?
No. Chartbeat found chatbots account for less than 1 per cent of all publisher page view referrals; Nieman Lab's own calculation was 0.7 per cent. ChatGPT page views to publishers rose more than 200 per cent between December 2024 and December 2025, but from a very small base. Akamai's research reached the same conclusion from another angle, finding AI chatbots delivered 96 per cent less referral traffic than Google search in the fourth quarter of 2024. The practical reading is that chasing AI referral clicks cannot replace lost search traffic, so value has to be captured in the read itself.
How much does AI bot traffic cost publishers?
The IAB Tech Lab has estimated that publishers lose around $2 billion a year to AI bot traffic, reflecting programmatic and affiliate revenue that bot reads would have earned had they triggered monetisation. Akamai describes the effect as eroding advertising and subscription revenue while raising infrastructure costs. The cost figures are modelled estimates rather than measurements, and they depend on assumptions about how much bot traffic would otherwise have been human, so quote them as estimates.
Are publishers blocking, and does it work?
Blocking is common but patchy. Known Agents data cited by the Press Gazette found publishers block only around 21 per cent of AI scraper bots through robots.txt, while AI scrapers and data providers honour robots.txt 95.6 per cent of the time. The infrastructure layer is moving towards default controls: Cloudflare said that from 15 September 2026 it would block mixed-use crawlers, which gather data for both search and AI training, from ad-supported pages for new customers and new sites on existing accounts, and it continues to pilot pay-per-crawl. Blocking protects content but earns nothing and removes a publisher from AI answers.
What do the numbers mean for strategy?
The data describes one shift: human visits that browser advertising monetises are declining, and machine reads that it cannot monetise are rising. Three responses follow. Measure AI traffic at the server or CDN, because client-side analytics cannot see it. Stay citable in AI answers so the brand keeps its visibility as clicks fall. And monetise the reads directly, since that is the only mechanism that earns from bots that never run JavaScript. blankspace works on the measurement and monetisation parts, classifying AI traffic at the CDN edge so Live Search Agent retrievals can be monetised.
These figures come from published reporting by Imperva, Cloudflare, HUMAN Security, 51Degrees, TollBit, Akamai, Chartbeat, Ahrefs and the IAB Tech Lab through September 2026. The underlying data changes often, so check the original sources before citing them externally.
Frequently asked questions
What percentage of internet traffic is bots in 2026?
Imperva's 2026 Bad Bot Report put automated traffic at more than 53 per cent of web traffic in 2025, up from 51 per cent a year earlier. Cloudflare Radar showed bots at 57.4 per cent of requests for HTML content over the seven days to 5 June 2026.
How much of web traffic is AI bots?
51Degrees found AI bot traffic rose from 8 to 13 per cent of web sessions in the year to May 2026, across more than three billion visits. Other datasets give different shares because they measure requests, sessions or HTML fetches, so always check the metric before comparing.
What is a crawl-to-refer ratio?
It is the number of pages an AI operator crawls for each referral visit it sends back. 2025 Cloudflare Radar figures ranged from about 14 to one for Google to tens of thousands to one for Anthropic, which shows content being consumed heavily while few readers are returned.
Do AI assistants send publishers meaningful traffic?
Not yet. Chartbeat found chatbots account for less than 1 per cent of publisher page view referrals, and TollBit measured click-through from AI applications falling to 0.27 per cent in the fourth quarter of 2025. ChatGPT referrals are growing quickly but from a small base.
How much do publishers lose to AI bot traffic?
The IAB Tech Lab has estimated around $2 billion a year in lost publisher revenue. This is a modelled estimate of revenue that bot reads would have generated if they triggered monetisation, so it should be quoted as an estimate rather than a measured loss.