Agentic search will, at least in theory, spell doom for many of the billions of sites on the open web, and usher in a strange back-end micropayment marketplace where agents trade commissions piecemeal. But is that theory undervaluing the power of people and the publishers who know how to connect with them?
Now that Google wants search and Gemini to become even more personalized for the average person, large-scale publishers need to figure out how to monetize the agentic era’s robot crawlers, especially as they continue to deal with the open web’s collapse.
Photo: Kevin Dietsch/Getty Images
Earlier this week, on a sunny California morning outside Google’s Mountain View headquarters, C.E.O. Sundar Pichai stood onstage at the company’s annual developers’ conference and offered an unsurprising but nevertheless startling declaration. He proudly touted that search, the business on which the company’s $4.6 trillion market cap was built 30 years ago, was getting an update for the A.I. future. Longtime head of search Liz Reid followed up with the details: Google A.I. will now attempt to personalize each search prompt with follow-up questions, and throw all of the relevant information into an interactive on-page box with a roundup of links on the side.
In other words, a question like “where is the World Cup” might yield a result that ties in a user’s Google Maps, Google Wallet, YouTube, and previous search history as it becomes “where is the World Cup, what are some of the lowest ticket prices, and which New Jersey transit option is the best for me if I’m coming from my apartment in Brooklyn.” What once spurred 10 blue links to other websites will now surface more YouTube Shorts and ads. Technologists cheered. Publishers passed the antacids once again.
This was further confirmation that we’ve fully entered the era of Google Zero, a term coined by The Verge’s Nilay Patel in 2024 to describe the looming death of search traffic. Google, Facebook, and X have all moved away from sending traffic out to the sea of websites that power the open web in an effort to grow their own walled gardens, evolving from advertising intermediaries to something like publishers themselves. The shift in strategy contributed to Alphabet’s nearly $90 billion in services revenue last quarter alone, driven by a near 20 percent growth in search.
Publishers, most of which have already spent the better part of the past decade on the back foot, are pivoting once again. Jonah Peretti, who recently announced he was selling BuzzFeed to Byron Allen, has said that most of the growth from loyal readers now comes from user-generated games. The New York Times is growing, largely based on its bundled games and cooking extensions. At CNN, Mark Thompson’s response to the collapse of cable and the open web is a weather app. James Murdoch’sacquisition of Vox Media and New York magazine seems largely about the podcast network, and the potential for its affinity-driven model to slot into his growing thought-leader events portfolio. In a TBPN interview last week, Condé Nast C.E.O. Roger Lynch grabbed some easy headlines with his revelation that he’d recently told his underlings to act as if search were already fully dead. (Good luck with all that, Chloe, Mark, and Adam!)
The past 18 months have been particularly telling. By mid-2025, the number of queries that resulted in zero clicks had increased from 60 percent at the beginning of the year to nearly 70 percent, while the percentage of traffic referral to Google Gemini and A.I. Overviews exploded. Between April 2025 and April 2026, traffic to Gemini’s website grew by more than 570 percent, per SimilarWeb.
Now that Google wants search and Gemini to become even more personalized for the average person, large-scale publishers need to figure out how to monetize the agentic era’s robot crawlers, especially as they continue to deal with the open web’s collapse.
Writing for Robots
Building for those crawlers is a daunting and novel task. Most websites are not going to see traffic bounce back to the height of referral in the mid-2010s. That means finding new ways to monetize a smaller pool of readers while also maximizing non-human interfacing opportunities within these new Google (or ChatGPT, or Perplexity, or Claude) boxes. Audience developers have tried to game algorithms in the past through search engine optimization, but the new world order requires trying to predict the follow-up to an initial prompt, and then monetizing off owned-and-operated platforms.
A.I. executives know they have to figure out these back-channel revenue structures with publishers and website operators. In a recent interview with The Atlantic’s Nicholas Thompson, OpenAI C.E.O. Sam Altman said he believes the future is… micropayments, arguing that publishers can set prices for both publication and author access, so that eventually an agent that wants to read The Atlantic has to pay a variable price (“$0.17 for a summary or $1 for a full article,” per Altman).
It’s not a stretch to imagine how much this could benefit the connectors of agents and publishers compared to just publishers alone. Micropayments have never worked for news publications because they’re so unpredictable and unreliable—and a one-time $1 transaction doesn’t encourage the same loyalty that comes with an annual $80 fee. In this scenario, companies like OpenAI or Google could theoretically also collect a toll on these payments, taking a small percentage of that micropayment fee, similar to what Amazon does as a third-party distributor of streaming services.
Perhaps the most optimistic take on the micropayments option is that plenty of companies are investing in this space for publishers. Cloudflare C.E.O. Matthew Prince, who recently said at a Semaforconference that “everything that’s wrong with the world today is Google’s fault,” implemented a “pay-per-crawl” service that would impact up to a quarter of total websites on the internet. New companies like TollBit and ProRata have created digital tolls and new compensation plans for publishers depending on how often they’re scraped by agents. If agents are seeking out and pulling in information billions or even trillions of times a day, as Sundar boasted onstage, then maybe the pure scale of robotic activity could make some form of micropayments work.
But A.I.’s real threat to the value of news is in pulling in aggregators from different sources and breaking through the paywall, allowing users to just train an agent to find a summary from a secondary source. Traffic to Google’s Gemini alone has seen staggering growth over the past year as Google places greater emphasis on integrating its leading A.I. model into more of its ecosystem.
Part of the question in this scenario is whether publishers want to solely sit in a supplier position, striking significant licensing deals with large language model companies that renew after an agreed-on period of time, or expand with the new readership habits formed by agentic use. So far, we’ve mostly seen the former. The Times, Dow Jones, Axel Springer, Condé Nast, and plenty of others have struck licensing agreements with OpenAI and Amazon. This is, in part, because it’s an understood business practice. Building out in-house teams that can focus on developing content and audience strategies around machine-readable content, and taking in swaths of data from L.L.M. partners to better understand what people are searching for and how it’s different from traditional search, is a larger and far riskier investment. Mainly because it doesn’t always work.
Don’t Count Out Humans
But let’s not totally ignore the actual humans here. We haven’t been completely replaced—not yet anyway. The open web is collapsing, but I’m hearing that New York magazineis on track to end the year with more subscribers than ever, and TheNew York Times is clearly proving that a strategy built around a newsroom can work. However, the pace of growth is likely to change, and executives quietly acknowledge this new reality.
Successful ventures must double down on two factors that A.I. can’t solve for in the foreseeable future: networking and obsessive analysis. One strategy is investing in talent to reliably keep audiences engaged; another is building products for those audiences that A.I. tools can’t replicate. ChatGPT and Gemini will eventually be able to create a hyperspecific portfolio dedicated to investments and companies that a user wants to keep an eye on, but it can’t create a space for interested audiences to gather, communicate, and interact. X might be able to solve this issue for the A.I. community, who can follow Altman or tune into Marc Andreessen’s Monitor the Situation or TBPN for a similar experience, but there are plenty of hyper-obsessive niches where community and communication become what audiences pay for each month.
For all that A.I. can do extremely well, knowing what people want to know after the initial query is sent, and contextualizingthat around emerging trends, is still outside most models’ purviews. As Ryan Broderick wrote earlier this week: “We might lose social and search by the time this is all over. Which means that even the chatbots that survive the A.I. bubble will have trouble scraping the public web for new information.”
Ironically, in this new A.I., agentic-first web era, websites and information are needed more than ever. Authority in those sites is also key: Less than 10 percent of adults in the U.S. use A.I. chatbots for news. And among those who do, about 50 percent said they don’t trust the results, per Pew Research. A new internet era is once again changing the foundational underpinnings, but it still demands useful, verifiable information. In fact, it’s never needed it more.
Shared with you by a Puck member
Enjoy this free article thanks to a friend. You can keep exploring Puck with a free account or enjoy a 14 day free trial.
Didn't get an email? Check your spam folder and confirm the spelling of your email, and try again. If you continue to have trouble, reach out to fritz@puck.news.
Didn't get an email? Check your spam folder and confirm the spelling of your email, and try again. If you continue to have trouble, reach out to fritz@puck.news.
Member Exclusive
Get access to this story
Create a free account to preview Puck’s full offering, including exclusive articles, private emails from authors, and more.