A disclosure law changes what a bot is allowed to claim, not what a bot is able to do. The UK's Automated Online Software (Access and Transparency) Bill, the US Stealth Bot Prohibition Act and a New York bill all rest on one idea: operators that copy content from websites should have to say who they are and why. That gives publishers a legal lever against bots that hide. It does nothing, on its own, to tell a genuine crawler from a convincing impersonation at the moment a request arrives.
What does the UK bot disclosure bill actually do?
The Automated Online Software (Access and Transparency) Bill is a Private Members' Bill brought by the Conservative MP Damian Hinds, drafted with the News Media Association (NMA) and presented in June 2026, according to InPublishing. It would require operators of bots that systematically access and copy content from UK websites to identify themselves, give a contact method and state their purpose, such as search indexing or AI training.
The bill is deliberately narrow. Hinds has said it "doesn't seek to ban anything", and the sponsors say it would not regulate AI models, change copyright law or dictate how scraped data may be used. Its aim is what Hinds calls "a basic principle of transparency": publishers should be able to see who is accessing their work and then decide whether to block, allow or negotiate.
Press Gazette reports that the proposed enforcement toolkit has four parts:
- A "white list" that compliant operators can join.
- Compliance notices that Ofcom could publish, naming offenders.
- The right for website owners to seek an injunction or damages.
- Court orders requiring UK internet providers to block persistent offenders.
The second reading was scheduled for Friday 16 October 2026. At the time of writing none of the coverage reviewed reports the outcome, and a private member's bill has a much better chance of passing with government support.
How does the US Stealth Bot Prohibition Act compare?
The federal bill, H.R.9915, is bipartisan, with Representatives Laurel Lee, Valerie Foushee and Gus Bilirakis named as sponsors. Playwire describes it as requiring bots, including AI crawlers, to disclose their identity and purpose, without banning crawling or mandating licensing deals. It would also replace a growing patchwork of state laws with a single national standard.
New York has moved first at state level. According to Press Gazette, the state's Stealth Crawler Prohibition Act has passed both the Senate and the Assembly and awaits the governor's signature. The News/Media Alliance, which backs the federal bill, helped draft and promote the New York measure.
The three efforts share a design: disclosure first, no restriction on AI itself. InPublishing describes the UK bill as closely mirroring the US one, and the publishers lobbying for each present it as a joint transatlantic push. For a deeper look at what a stealth crawler is and how these bills define the problem, see our guide to stealth crawlers.
Why are publishers pushing for legislation now?
Voluntary signals have run out of road. In a letter reported on 16 October, the heads of eight UK national newspapers, joined by Reddit, Mumsnet and Condé Nast, told the Prime Minister that "voluntary standards have failed" and that robots.txt "carries no legal force and is being ignored on a mass scale".
The NMA says 19 of the 20 largest UK news websites can be scraped by stealth bot tools on sale today, despite paywalls and bot-blocking technology. Digiday's reporting from the US side tells a similar story: the Financial Times' Fiona Spooner described an account attempting to pull about one million articles in eight hours from roughly 700,000 IP addresses, and the same briefing reports more than 100 million successful scrapes of Reach content over six months.
Publishers also want leverage, not just protection. The News/Media Alliance's Danielle Coffey has argued that identification requirements could help publishers control access and negotiate licensing with AI companies, because a buyer that must declare itself can be invoiced.
Do bots already lie about who they are?
Yes, and the scale matters for how much a disclosure law can achieve. Centinel Analytica examined requests to news and media sites it protects between late June and 21 September 2026, verifying each claimed crawler identity against the operator's documented method: a signed request where supported, then published IP ranges, then reverse and forward DNS.
On the typical site, the share of requests that failed verification was high for several well-known names: 84 per cent of GPTBot claims, 99 per cent of Perplexity-User claims and 14 per cent of Googlebot claims. Measured across all traffic, the figures are far lower for GPTBot (2 per cent) because genuine crawler volume concentrates on a few busy sites. Centinel also found that 94 per cent of fake Googlebot requests copied Google's real current version string, and that failed requests overwhelmingly came from data centres.
Two points follow. First, a user agent string is a claim, not evidence. Second, these impersonators are not "undisclosed" in the way the bills imagine; they disclose a false identity. A law that says "tell the truth about who you are" gives you recourse against the liar after the fact, but it does not change what the request looks like in your logs.
Where will disclosure laws fall short?
Three limits are worth planning around.
Jurisdiction. Candr Media's Chris Dicker told Digiday that "Cloudflare only governs what's behind Cloudflare", and that well-funded scrapers can route around controls through residential proxies or buy scraped data from someone else. A UK statute binds operators within UK reach; an anonymous operator elsewhere is harder to find, never mind sanction.
Verification. None of the coverage reviewed describes a technical mechanism for proving a disclosure is true. The UK white list implies some registry, but a registry of declared operators only helps when requests can be tied to an entry. This is the gap that cryptographic approaches such as Web Bot Auth are designed to fill.
Resale. Publishers say AI companies often buy content from scraping firms rather than from publishers. Disclosure by the scraper does not automatically trace the content downstream to the model that ultimately uses it.
What should publishers do while the bills are pending?
Treat the legislation as a tailwind, not a control. People Inc.'s Jon Roberts told Digiday that enforcement "has to be the CDN level", and that position is hard to argue with: the only place a request can be classified before it is served is at the edge, where network signals, signed identity and behaviour can be checked together rather than trusting a header.
Practically, that means four steps:
- Separate claimed identity from verified identity in your logs, using the verification order above.
- Decide per verified operator whether to allow, charge or block, rather than per user agent string.
- Keep a record of requests that fail verification, since evidence of deception is what any future statutory remedy will need.
- Watch how the UK second reading and the New York governor's decision land, then revisit your terms of access.
Edge-level classification is also where traffic can be monetised rather than merely refused. blankspace operates at that layer, detecting live search agent traffic at the CDN edge so that publishers see what is visiting before deciding what it is worth. For the monetisation side, see how to monetise AI crawler traffic.
Frequently asked questions
Is the UK bot disclosure bill law yet?
No. It is a Private Members' Bill that was scheduled for second reading on 16 October 2026, and the coverage reviewed does not report the outcome. Such bills rarely progress without government backing, which is why eight newspaper chiefs wrote to the Prime Minister asking for support or a government bill.
Does the Stealth Bot Prohibition Act ban AI crawling?
No. As described by Playwire, H.R.9915 would require bots, including AI crawlers, to disclose their identity and purpose, and does not ban crawling or require licensing deals. Enforcement and penalty provisions were not described in the coverage reviewed, so read the bill text before relying on it.
Would a disclosure law replace robots.txt?
No. Robots.txt expresses a publisher's preferences and carries no legal force, which is the publishers' central complaint. A disclosure law would make it possible to know who is reading those preferences, but a publisher would still need a technical way to apply them.
Can publishers tell whether a bot's disclosure is true?
Not from the user agent alone. Verification needs a signed request, a published IP range or a reverse and forward DNS check, as in Centinel's method. Without one of those, a declared identity is only a claim.
Do these bills help publishers get paid for AI use?
Indirectly. The sponsors say the bills do not mandate licensing, but publishers argue that knowing who is scraping is a precondition for negotiating with them. Payment still depends on commercial agreements or on infrastructure that prices access.