Publishers Are Rebuilding the Web So AI Agents Can Read It. The Page That Paid the Bills Doesn't Come Along.

Time is tearing its own website down to the studs and rebuilding it for readers who never load the page. On July 1, Digiday reported that the century-old publisher is converting all of its webpages from HTML to markdown, the stripped-down text format AI models read cheaply. First it blocked AI bots by default. Then it built whitelists for the ones it approves.

Mark Howard, Time’s chief operating officer, described what an agent actually takes when it visits:

“Just getting the content itself and the metadata, but they’re not getting the full page experience.”

The full page experience is the part that earns. The ad, the affiliate link, the newsletter prompt, the related-stories rail. Strip a page down to what a model can parse in a quarter of a second and you’ve deleted the surfaces the business was built on.

That trade is exactly what the new tooling is built to make. TollBit, the startup that sits between publishers and AI crawlers, says a markdown page returns to a model in 0.25 seconds against the minute-plus it can take to scrape raw HTML. It also clocked token reductions of up to 97.5% across publishers including Newsweek and the Mirror. Fewer tokens means a smaller inference bill for whoever runs the agent. As TollBit puts it, every unnecessary div, span, or script tag is a token the model has to pay for.

Toshit Panigrahi, TollBit’s co-founder and CEO, framed the payoff for publishers in the same currency:

“AI can comprehend more of your article because they’re not spending money parsing.”

From workaround to web standard

None of this stays a bespoke hack for long. Google and Microsoft are co-developing WebMCP, a proposed standard now drafted through a W3C community group, that lets a site expose its functions as tools an agent can call directly rather than clicking through a page. Google shipped a Chrome beta in February. Cloudflare will convert a site’s HTML to markdown in real time the moment an AI system asks for it. André Cipriani Bandarra, a developer relations engineer at Google, put the ambition plainly:

“As the agentic web evolves, we want to help websites play an active role in how AI agents interact with them.”

Not everyone reads that ambition as good news. Scott Messer, principal at the advisory firm Messer Media and a former media executive, is blunt about what the rebuild removes:

“By intentionally streamlining the web for AI agents, we are stripping away the very things that make the open web profitable.”

Why this lands on marketing, not just publishing

The open web’s money model assumes a human loads a page, sees an impression, and maybe clicks. An agent that fetches clean markdown and never renders the page breaks every link in that chain: the display impression never fires, the retargeting pixel never sets, and the funnel that used to start with a click has no click to start from. The reader still gets your facts. Your business gets nothing it used to charge for.

That’s why the discipline replacing search-era SEO is generative engine optimization, the work of landing inside the AI answer rather than in the list of blue links. Publishers are already triaging their pages for it. Le Monde is testing whether an arriving agent represents a paying subscriber. The Economist is starting its agent-readable rebuild with marketing copy and B2B sales material, the pages sitting closest to revenue.

The uncomfortable part for any brand is that the move making you legible to a machine also makes you generic to a human. A markdown feed carries your words, not your design, your voice, or the owned experience that was supposed to be the moat. Discovery by agent is discovery without a storefront.

So treat the agent-ready web as a distribution channel with its own economics, not a technical chore to check off. Three moves follow. Decide which pages you actually want machines to read cheaply, and don’t convert the rest by default. Instrument agent traffic separately from human traffic, because a metric that blends them will lie to you about what’s working. And settle internally what a “visit” is now worth before finance settles it for you, because the click you used to bill against is being quietly retired. The web is growing a second audience that buys nothing it sees. Build for it with your eyes open.