Most publishers think of an AI licensing negotiation as a copyright question: who owns the content, and what is it worth. The case now running in the Southern District of New York asks something different and more uncomfortable. It asks what happens when a platform uses a technical file that every crawler on the web is expected to obey to say yes to one company and no to everyone else. That is not a copyright question at all. It is a competition question, and the answer determines whether an exclusivity clause is a premium worth charging for or a liability sitting in your contract.
The short answer is that selling AI access is lawful, selling it exclusively is usually lawful, and selling it exclusively becomes legally dangerous at the point where you have market power in something a buyer cannot get elsewhere and your exclusivity forecloses rivals from competing. Almost no publisher is anywhere near that line. A handful are, and the ones that are tend not to think of themselves as dominant in anything.
What is actually being alleged against Reddit
On 28 August 2026, SerpApi, LLC filed a 63-page answer and counterclaims to Reddit's amended complaint in the United States District Court for the Southern District of New York, Case No. 25-cv-8736, before Judge Paul A. Engelmayer. PPC Land reported the filing in detail on 2 September 2026, and the legal trade title MLex covered it under the framing that SerpApi accuses Reddit of monopolising its own users' content.
Alongside eighteen defences, the filing adds six counterclaims: monopolisation and attempted monopolisation under Section 2 of the Sherman Act, plus four counts seeking declaratory judgment. The remedies sought are the part publishers should read twice. SerpApi asks for treble damages under Sections 4 and 16 of the Clayton Act, and for an injunction against two specific things: requester-specific robots.txt directives, and exclusive crawl agreements.
The factual core is narrow. According to the pleading, Reddit serves one robots.txt file to Google permitting crawling of reddit.com, and a different file to every other search engine and agent prohibiting it. The split is dated to 1 July 2024, following a notice posted by Reddit Chief Legal Officer Ben Lee in the r/redditdev community on 25 June 2024. The counterclaims name the casualties: Microsoft's Bing, DuckDuckGo, Mojeek and Qwant lost the ability to index new Reddit conversations. Mojeek chief executive Colin Hayhurst is quoted in the pleading describing the effect as "killing everything for search but Google."
Three days after the change, on 4 July 2024, a managing director at the technical consultancy Merj used Google's own Rich Results testing tool to fetch Reddit's response as Googlebot actually receives it, and published a finding that a different, more permissive file was being served. That test, run with a free public tool, is now cited in a federal pleading. It is worth pausing on how easy the evidence was to gather.
The commercial backdrop is a Google partnership announced on 22 February 2024, the same day Reddit filed its S-1, reported at 60m dollars per year. Nothing in the counterclaims has been tested. Reddit has not filed a response. A case management conference is scheduled for 1 October 2026.
Why exclusivity is the legal problem, not the licensing
Nobody in this case is arguing that Reddit cannot charge for access. SerpApi General Counsel Chad Anson's public statement makes the distinction explicitly, arguing that Reddit's users own what they write and that Reddit's own policies told them it would be shared publicly and freely.
The antitrust theory does not turn on the price. It turns on three things stacked together. First, a defined market in which the seller has power. The counterclaims plead a market for large-scale, authentic human conversational data in threaded discussion forums, used for search indexing and for training and grounding AI systems, and they build the case for that market out of Reddit's own marketing: a chief financial officer describing the platform as "1-of-1", and chief executive Steve Huffman stating that large language models "would not exist as we know them without Reddit." A seller's own investor deck is the most useful document an antitrust plaintiff can find.
Second, conduct that forecloses competitors rather than merely favouring a customer. Granting Google a licence is favouring a customer. Configuring your server so that Bing, DuckDuckGo, Mojeek and Qwant receive an instruction to stay out, while Google receives one that lets it in, is the kind of fact pattern that gets characterised as foreclosure.
Third, harm to competition rather than to one competitor. This is where the presence of four named search engines matters more than the presence of SerpApi.
Stack all three and you have a Section 2 claim. Remove any one and you usually do not. Most publisher licensing deals fail the first test comprehensively, which is the reassuring part of this analysis for the overwhelming majority of the market.
The six-month gap and why it changes the character of the case
Reddit's surviving claims rest on Google's SearchGuard, the JavaScript challenge system that separates human visitors from automated software, characterised in the litigation as a technological measure protecting copyright.
SerpApi's answer identifies a chronology problem. By Reddit's own account, no purported technological control measure existed between 1 July 2024, when non-Google crawlers were locked out, and January 2025, when SearchGuard was deployed. That leaves roughly six months in which the exclusion operated with no copyright-protection mechanism behind it at all. The sixth counterclaim asks the court to declare that no cognisable claim exists under the Digital Millennium Copyright Act for any period before January 2025, grounding the request in a concession Reddit's counsel made at oral argument.
Chief executive Julien Khaleghy put the same argument in one line in a blog post on 31 August 2026: if the case had been about protecting copyright, "the protection would have come first. It didn't."
Publishers should recognise the structure of that argument, because it applies to a lot of crawler-blocking infrastructure built over the past two years. If you block a crawler and then later describe the block as protecting your copyright, the sequence of events is discoverable and the gap between the two is an argument someone else gets to make. Blocking for commercial reasons is entirely legitimate. Describing a commercial block as a copyright measure after the fact is where the exposure sits.
The courts are already reading this two ways
Two federal courts are currently reaching different conclusions on closely related conduct. On 20 July 2026 Judge Yvonne Gonzalez Rogers dismissed every claim in Google's parallel scraping case against SerpApi in the Northern District of California, finding that SearchGuard guards advertising revenue rather than copyright. In New York, Judge Engelmayer allowed Reddit's core anti-circumvention theory to proceed. On 31 July 2026 the New York court did dismiss three of Reddit's counts against SerpApi, leaving the anti-circumvention and trafficking counts and civil conspiracy.
Behind both sits an older line of authority that cuts against platforms. In X Corp. v. Bright Data Ltd., dismissed in the Northern District of California in May 2024, the court warned that allowing a platform to control public data on its own terms would let it "yank into its private domain and hold for sale information open to all", entrenching a private copyright system alongside the statutory one. That reasoning, and hiQ Labs v. LinkedIn before it, protects scrapers. It also constrains publishers, which is why the counterclaims cite it.
Regulators are asking the same question from the other end
The publisher-side version of this issue is already under formal investigation in Europe, and it is worth understanding that the regulators are looking at the mirror image of the Reddit claim.
On 9 December 2025 the European Commission opened an Article 102 investigation into whether Google abused a dominant position by using web publisher content and YouTube uploads for AI purposes. Among the Commission's stated concerns is whether Google granted itself privileged access to content in a way that disadvantages rival AI developers. In February 2026 the European Publishers Council, whose members include DMG Media, The Guardian, News UK and The New York Times, filed a formal antitrust complaint over AI Overviews and AI Mode. Neither proceeding has concluded and opening an investigation prejudges nothing.
The point for publishers is the shape of the theory rather than the outcome. Regulators have accepted the premise that privileged access to content can be an antitrust harm. Once that premise is established against a platform, it is available against a publisher who arranges the same outcome from the supply side.
What this means if you are weighing an exclusive deal
Exclusivity is a thing buyers pay for, and there is no reason to give it away. But the risk is asymmetric and badly understood, so five things are worth knowing before the clause goes in.
Most existing deals are not exclusive. Axel Springer's agreement with OpenAI, reported at around 13m dollars per year over three years, was described at the time as committing neither side to exclusivity, and that structure has been the pattern for most of what followed. If a buyer asks you for exclusivity, they are asking for something the market does not currently treat as standard, and they should be paying a premium that reflects that.
Exclusivity is worth less than it looks. A deal with one AI company resolves one relationship. It does nothing about the long tail of agents, retrieval systems and stealth crawlers that will read your pages regardless, and it removes your ability to sell the same access again.
You probably do not own all of it. Reddit's exposure here is amplified by a structural fact: under its user agreement users retain ownership of their content and Reddit holds a non-exclusive sublicensable licence, and the counterclaims put user-generated material at more than 99.99% of the corpus. Any publisher selling exclusive access to contributor content, forum content, comments or syndicated wire material should check what it actually has the right to sell exclusively, separately from what it has the right to sell.
Promises made publicly become evidence. The counterclaims lean heavily on Reddit's own public statements, including a promise that the Internet Archive would continue to be allowed to crawl, followed by the archive being cut off in August 2025. Whatever your access policy says, the operative version is what your server does.
Serving different instructions to different requesters is the specific conduct at issue. Requester-specific robots.txt directives are the thing SerpApi is asking a court to enjoin. Differentiating by user agent is completely normal and mostly uncontroversial. Differentiating in a way that admits one commercial partner and excludes that partner's competitors is the pattern under challenge, and it is trivially detectable by anyone with a free testing tool.
Why non-exclusive monetisation carries less of this risk
There is a straightforward structural reason why per-request, non-exclusive monetisation attracts less legal risk than exclusive licensing: a mechanism open to all buyers on the same terms cannot foreclose anyone. Pay-per-crawl arrangements, collective licensing schemes and retrieval-layer advertising all share that property. They price access rather than allocate it.
That is also the honest argument for treating exclusivity as the exception rather than the goal. blankspace operates at the CDN edge and injects contextual brand mentions into AI answers on a per-request basis, which means every agent that arrives is monetised on the same terms and none is excluded to benefit another. That is one answer among several to the same question, and the general principle matters more than any particular implementation of it: revenue models that scale across all buyers avoid a category of legal exposure that revenue models based on exclusion do not.
It is also worth being clear that the alternative to exclusivity is not blocking. PPC Land has reported research finding that news publishers who blocked generative AI crawlers saw a 23.1% decline in log-monthly visits. Exclusion is expensive whether you do it for one buyer's benefit or for your own.
Frequently asked questions
Is it illegal for a publisher to sell exclusive AI access to its content?
No, not as a general matter. Exclusive licensing is a normal commercial arrangement and most publishers can sign one without any competition law concern, because antitrust liability under Section 2 of the Sherman Act requires monopoly power in a defined market plus conduct that forecloses competition. A regional news site or a specialist trade title will not have that power. The risk concentrates in platforms holding a corpus that buyers genuinely cannot substitute, which is a very small group.
What is a requester-specific robots.txt file and why is it a legal issue?
It is a server configuration that returns different crawling instructions depending on who is asking, so one crawler is told it may index a site while others are told they may not. It is at issue because SerpApi's counterclaims against Reddit, filed on 28 August 2026, ask the court to enjoin the practice as a mechanism for excluding a commercial partner's competitors. Differentiating by user agent in a single published file is ordinary practice and is not what is being challenged.
Does an exclusive AI deal stop other AI companies using my content?
In practice, no. An exclusivity clause binds you and the counterparty, not the wider market of crawlers, agents and retrieval systems that will continue to read your pages. It commits you to refusing other buyers while doing nothing to prevent unlicensed access, which is why exclusivity generally transfers value to the buyer rather than creating it.
Can I sell exclusive access to content my users or contributors wrote?
Only to the extent your terms actually grant you that right. The Reddit counterclaims highlight the problem directly: user-generated material is pleaded at more than 99.99% of the corpus, while Reddit's user agreement leaves ownership with users and gives Reddit a non-exclusive sublicensable licence. Publishers with forums, comments, contributor networks or syndicated wire content should confirm what they hold before agreeing to any exclusivity term.
When will the Reddit and SerpApi case be decided?
There is no decision date. The counterclaims were filed on 28 August 2026 and Reddit had not responded to them at the time of writing, with a case management conference scheduled for 1 October 2026. Nothing in the pleading has been tested and antitrust cases of this kind typically take years. Publishers should treat the filing as a signal about how exclusivity arrangements will be scrutinised rather than as settled law.