AI Agents Now Make Most of Your Web Traffic, and Referral Clicks Stopped Counting Any of It

AI agents made 17.7 billion requests to the sites in DataDome’s network between April and June, a 45% jump from the quarter before. Almost none of it sent a human anywhere. Over the same stretch, ChatGPT-User, the crawler that fetches pages to answer questions, visited websites 6% less than it had in the first quarter.

The bots are reading more of your content and sending fewer people to it. That breaks the number most marketing teams still use to judge whether AI is helping them: referral traffic, the clicks that arrive from an AI tool. Digiday laid the data out this month, and the takeaway is blunt. The scoreboard went blind to the channel it was supposed to measure.

The click was always a stand-in for attention. The attention moved inside the answer. When a reader asks ChatGPT or Google’s AI a question, the model reads your page, writes the response, and the reader never arrives. Grading that by referral clicks is like judging a library by how many books get carried out the door.

So what replaces it. The honest metric now is whether the answer mentions you at all, and whether it links back when it does. Those are two different things, and both are rare. Webflow built an AEO Maturity Index across 2,000 websites and found the median company showed up in just 16% of the AI answers where it was relevant, and earned a cited link only 6% of the time. When a brand did get mentioned, the AI got its facts wrong about a third of the time.

“We are in early days in a new medium,” Guy Yalif, chief evangelist at Webflow, told Digiday. The company’s data suggests the medium already rewards size: large companies posted mention rates twice as high as everyone else, and share of voice 80% higher.

That last point is the one that should worry a small team. The old organic-search game had a long tail, where a sharp post from a 20-person company could outrank a giant on a specific query. We covered how AI Overview visibility and Google rankings have come apart, and how the organic click is collapsing. The new game concentrates attention on brands the model already trusts, and trust tracks authority, which tracks size.

The reflex response has been to bolt on a technical fix. The most popular one is llms.txt, a file that tells AI crawlers how to read your site. Adoption jumped 8.8 times in a year, from about 4,000 sites to 36,000. Then Ahrefs pulled the server logs and found that 97% of those files were never requested by anything in May. Jon Gillham, CEO of Originality.ai, whose team tracked the adoption curve, put it plainly: “adoption isn’t usage. If 97% of these files are never requested, this isn’t an AI visibility strategy yet.”

Here is the mechanism the traffic numbers hide. An AI answer engine crawls to build an index and crawls again to keep it fresh, so the same page gets fetched over and over while the reader stays inside the chat. Jérôme Segura, VP of threat research at DataDome, said the shift is toward permanence: “As AI traffic becomes more persistent, publishers need to set guidelines for crawling and indexing like they have been doing for Google.” The visit is the crawl. The payoff, if there is one, is being the source the answer trusts enough to cite.

Blocking the crawlers does not fix it either, and it may make things worse. HasData found that 56% of news publishers now block at least one AI crawler in robots.txt, and half block GPTBot specifically. But of the sites that ban GPTBot, nearly 40% still served it a page anyway, because a block only works on a bot that agrees to obey it. Roman Milyushkevich, CEO of the web-data firm HasData, called the position a bind: “publishers are in a ‘change-or-die’ spot because their protection only works with polite bots.” Wall off the well-behaved crawlers and you lose the AI-search visibility they bring, without stopping the scrapers you actually fear.

For a marketing operator, three things follow. Stop reporting AI’s value as referral traffic, because the channel that matters no longer produces a click. Start tracking two numbers instead, how often you get mentioned in answers and how often that mention carries a link, and treat them separately. And skip the file-based rituals that make a dashboard look busy while nothing reads them.

The click was a proxy for attention, and the proxy broke. Keep measuring it and you will conclude AI is doing nothing to your traffic, right up until you notice that the answer everyone reads never says your name.