For two years the only lever publishers had against AI crawlers was a blunt one: let them in through robots.txt, or block them. Really Simple Licensing adds a third option in between. It is a machine-readable way to state the price and terms on which an AI company may use your content, written directly into the files crawlers already read, so the choice is no longer just free access or no access.
The standard was launched on 10 September 2025 and reached its official 1.0 specification in December 2025. It matters to publishers because it is the first serious attempt to turn the AI scraping problem into a transaction rather than a fight, and because the names behind it - and the names conspicuously absent from it - tell you exactly how far the idea has travelled and how far it still has to go.
What is Really Simple Licensing (RSL)?
RSL is an open, XML-based standard that lets publishers and platforms define machine-readable usage, licensing and legal terms for any digital asset on the web. It is built on RSS, the syndication standard that distributed blog and news content across the early web, and it is managed by the RSL Collective, a non-profit set up to administer the standard and to negotiate on behalf of the publishers who adopt it.
In plain terms, RSL lets a publisher say to an AI crawler: you may use this content, here are the conditions, and here is what it costs. Those terms are expressed in a format a machine can parse, so they can be applied automatically across millions of pages rather than negotiated deal by deal.
How does RSL work?
A publisher implements RSL by adding licensing terms to the files crawlers already check, principally robots.txt, and by referencing an RSL licence document that sets out the conditions in full. Instead of a robots.txt line that simply allows or disallows a given bot, the publisher attaches terms of use to that permission.
RSL supports a range of licensing and royalty models. A publisher can offer content for free, require attribution, demand a subscription, charge pay-per-crawl, or charge pay-per-inference. Pay-per-crawl compensates the publisher each time an AI application crawls the content. Pay-per-inference compensates the publisher each time an AI application uses that content to generate a response, which ties payment to the moment of actual use rather than to the act of fetching the page.
Two supporting protocols are designed to make the terms enforceable rather than merely stated. The Open License Protocol is an OAuth 2.0 extension for acquiring and validating RSL licences, and the Crawler Authorization Protocol is an HTTP authorisation scheme that licensed crawlers use to prove they have agreed to the terms. Together they are meant to let a server check, in real time, whether an incoming crawler holds a valid licence before it serves the content.
Who is behind RSL, and who supports it?
RSL was co-founded by Eckart Walther, a co-creator of the original RSS standard, and Doug Leeds, former chief executive of Ask.com and IAC Publishing. That lineage is deliberate: the pitch is that RSL does for licensing what RSS did for syndication, by turning an ad hoc practice into a shared, open format.
The early backers are substantial. Reddit, Yahoo, People Inc, Internet Brands, Ziff Davis, Quora, O'Reilly Media, Medium and the infrastructure provider Fastly were among the first to support the standard at launch. They were later joined by publishers including Arena Group, BuzzFeed, USA Today Co and Vox Media. The RSL Collective says more than 1,500 media organisations, brands and technology companies now endorse the standard, spanning tens of billions of web pages, which it frames as most of the high-quality content used to train AI foundation models.
How is RSL enforced?
This is where the standard meets reality. RSL can state terms, but stating terms is not the same as collecting money, and enforcement currently depends on where a publisher's content is served.
Infrastructure partners provide the teeth. Fastly acts, in its own description, as the bouncer at the door, checking whether an AI crawler has agreed to license content before granting access. That works for publishers served through Fastly's content delivery network. Cloudflare runs a parallel mechanism: its pay-per-crawl system uses the HTTP 402 "Payment Required" response to create an actual transaction, where the publisher sets a price, the AI company sees it, and it either pays or walks away. Cloudflare has also blocked AI crawlers by default for new domains since July 2025, which gives the request for payment some leverage.
For everyone else, the position is weaker. A publisher without an enforcing CDN can publish RSL terms and ask for payment, but has no technical mechanism to compel compliance. The standard describes the deal; the CDN at the edge is what makes the deal stick.
The catch: will AI companies actually pay?
The honest answer, as of mid-2026, is that the largest ones have not agreed to. No major foundation model developer has publicly committed to honour RSL 1.0. OpenAI, Google DeepMind, Anthropic, Meta, xAI, Mistral and Amazon have made no public commitment to read or respect the file, and several declined to comment when the standard launched. A licensing standard that the buyers have not signed up to is, for now, an asking price with no obligation to pay it.
There are deeper obstacles than reluctance. Pay-per-inference, the model publishers most want because it ties payment to genuine use, requires detailed logging inside the AI system to trace which sources shaped a given answer. Few models have that traceability built in, and adding it is engineering work that vendors have resisted. There is also a track record of evasion: Cloudflare reported catching Perplexity using stealth bots to route around crawler restrictions, which is a reminder that some companies will treat access controls as obstacles rather than agreements.
The likeliest path to compliance may not be voluntary at all. Several AI content lawsuits are still working through the courts, and a ruling that forces model builders to pay for training data would change the calculus overnight, turning a standard the industry can ignore into the cheapest available way to license at scale.
How RSL differs from llms.txt and from content licensing deals
It helps to place RSL against two things publishers already know. It is not llms.txt, which is a comprehension file that helps an AI find and understand your most important content but says nothing about payment. RSL is the opposite concern: it is about compensation, not comprehension, and the two can sit on the same site at the same time.
It is also not the bespoke licensing deal. The headline agreements between large AI companies and individual publishers are privately negotiated, restricted to a handful of the largest names, and closed to almost everyone else. RSL's ambition is to automate and democratise that arrangement so a mid-sized or small publisher can offer the same machine-readable terms a giant would, without a negotiating team. Whether the buyers turn up is the open question, but the design intent is to extend licensing to the uncompensated majority rather than the favoured few.
What RSL means for publishers
For publishers, RSL is worth implementing as a low-cost option that costs little to add and improves your position if the market moves toward paid access. It declares a price, it documents your terms if a dispute or a lawsuit later turns on them, and on an enforcing CDN it can begin collecting today. The realistic expectation is that it is an option you take out now rather than a revenue line you can bank on this year.
It is also worth being clear about what RSL governs and what it does not. RSL is a control on the input - whether a crawler may read and train on your content, and on what terms. It does nothing about the output, the moment an AI assistant or live search agent surfaces an answer to a user and a brand could be present in that answer. Those are two distinct layers of the AI value chain, and a publisher can sensibly act on both: license the read through RSL where it is enforceable, and capture the answer through contextual advertising in AI responses. This is the layer blankspace operates in, detecting live search agent traffic at the CDN edge and injecting contextual brand mentions into AI answers, which monetises the surfaced response rather than the underlying crawl. Licensing the input and monetising the answer are complementary, not competing, and the publishers who do best in the AI era are likely to be the ones who treat them as two parts of the same strategy.
Frequently asked questions
What does RSL stand for?
RSL stands for Really Simple Licensing. The name is a deliberate echo of RSS, Really Simple Syndication, and the standard is co-created by an RSS co-founder. The intent is the same: take a practice that previously required custom work and express it in a simple, open, machine-readable format that any site can adopt.
Is RSL free to use?
Yes. RSL is an open standard, and adding RSL terms to your robots.txt and licence files costs nothing in itself. The cost and complexity sit in enforcement and collection, which in practice depend on an infrastructure partner such as Fastly or Cloudflare to check crawler licences and handle payment at the edge.
What is the difference between pay-per-crawl and pay-per-inference?
Pay-per-crawl charges an AI company each time it crawls your content, so payment is tied to access. Pay-per-inference charges each time the AI uses your content to generate a response, so payment is tied to actual use in an answer. Pay-per-inference is the model many publishers prefer because it reflects real value, but it is harder to enforce because it requires the AI system to log which sources shaped each output.
Have any AI companies agreed to pay through RSL?
Not among the major model developers, as of mid-2026. OpenAI, Google DeepMind, Anthropic, Meta, xAI, Mistral and Amazon have made no public commitment to honour RSL 1.0. Support so far comes from publishers and from infrastructure providers such as Fastly and Cloudflare. Without buy-in from the AI companies that do the crawling, RSL states a price the buyers are not yet obliged to pay, which is its central limitation today.
Should publishers add RSL to their site now?
For most publishers it is a reasonable, low-cost step. It declares your terms, creates a documented record that could matter in future disputes or litigation, and positions you to collect if the market or the courts shift towards paid access. Treat it as insurance and option value rather than as a reliable revenue line, and pair it with a plan for monetising the AI answers your content already appears in, since RSL on its own only addresses the crawl.