Welcome back to What I’m Hearing+, live from Brooklyn. This afternoon, an inside look at the WGA’s groundbreaking data transparency deal—and the problem with trying to determine success via viewership alone.

Programming note: I’ll be in Las Vegas on Thursday and Friday for the American Bar Association’s Forum on Entertainment and Sports, where I’ll be presenting on the trials and tribulations of the streaming business. If you’re a fan of this email, you’ll enjoy a “live” version. Grab tickets here.

Let’s get started…

As the industry shapeshifted over the past decade—linear television faded, theatrical suffered, and all the biggest media companies overinvested in streaming—many have wanted to recapture the scorekeeping that once regularized the business, and for good reason. Nielsen wasn’t perfect—it didn’t capture the success-based metrics of a premium network like HBO, which followed a subscription model and wasn’t focused on viewership for advertisers—but it more or less worked across networks, shows, and audience demos. Furthermore, it also created an apples-to-apples benchmarking system that was necessary for talent, advertisers, and executives. No wonder that “transparency,” an impossibly vague term covering everything the streamers don’t want to share, became one of the most heated battles in the writers’ strike.

So what did the WGA and the AMPTP agree to in their 94-page deal? The contract states that writers will now receive bonuses of 50 percent of streaming residuals if a title is viewed by 20 percent or more of a platform’s domestic subscriber base. For Netflix, this would be about 14 million completed views in the U.S., based on an estimated 68 million subscribers. As my friends at What’s On Netflix have pointed out, about 50 percent of titles released between January and September of 2023 would have been eligible for that bonus.

The studio-streamers also must share the “total number of hours streamed, both domestically and internationally, of self-produced high budget streaming programs.” But what does that language actually mean? And will this deliver the information that writers (and Wall Street analysts) have been seeking?

Viewership, after all, is only one puzzle piece when it comes to determining the value of a show to a streamer. For instance, two similar titles can achieve the same level of viewership within the first 90 days, but one series might bring in more new, high-value customers (an audience that the streamer has invested in chasing, let’s say), while the other doesn’t acquire or help retain subscribers. And how should the cost of the series factor in the value calculus?

None of those questions will be directly answered by the data in this WGA deal. Of course, the current contract is just a blueprint of sorts—as union negotiator Adam Conover argued to my partner Matt Belloni on The Town last week, the most consequential impact of the deal is that it provides a framework for future renegotiations. So while the AMPTP isn’t yet offering to fully illuminate streaming’s black box, the box has been opened, at least just a little.

Here’s a closer look at what inside, the loopholes the streamers may use to limit transparency, and what writers could push for next. Alas, the complexities of the numbers and the weighting of their value suggest that the imperfect simplicity of the Nielsen era is a thing of the past.

Very few series perform at high levels across all categories. Between January and August this year, according to Parrot Analytics, where I work as director of strategy, the highest performing acquisition titles on Netflix in the U.K. included The Night Agent, Wednesday, Black Mirror, Shadow and Bone, Queen Charlotte, and Manifest. None of these titles should be a surprise to people reading this column. Some of the best performing titles for engagement, however, might be less familiar: Bojack Horseman, The Umbrella Academy, Altered Carbon, The Glory, and Inside Job.

What does that tell us? The first group of titles brought in the highest number of subscribers (and got a lot of views within the first 90 days). But many of those users likely binged one of those shows and then moved on to the next thing. The second group, however, demonstrates a strong ability to keep Netflix subscribers engaged on the platform longterm. This metric is different still from the top titles in the U.K. for retention, or the likelihood to prevent churn, which included Young Sheldon, Peaky Blinders, and Taskmaster. One way to think about it is that high acquisition titles get people to open the app in the first place; strong engagement titles make people think “I’ll keep watching Netflix.” These metrics are equally important for advertising partners, who haven’t won guarantees to the same information as WGA members, but who remain crucial to this conversation.

I know that the guild wants simplicity in its transparency, but streaming is a complex business. Strong engagement or retention titles, which are essential to enhancing the lifetime value of a subscriber, may not see their import reflected in overall viewership. Decay rates—the measure of declining engagement after 90 days compared to other titles in similar genres and cost brackets—also aren’t properly weighted. When thinking about the overall lack of benchmarks provided by the studios, it’s difficult to contextualize performance.

There are always exceptions—for example, Grey’s Anatomy and Friends are high retention titles for Netflix and Max, respectively, that chart each week on Nielsen in the U.S. They’re also both high retention and high engagement titles. But what happens if those titles are removed? Consider that 82 percent of Grey’s strength on Netflix in a region like Latin America is its ability to retain high risk churn subscribers, according to Parrot. That provides immense value to Netflix, which may not be captured by the metrics the platform shares. That sort of data is particularly important to measuring the success of originals. If a title is delivering strong value outside of what pure viewership can measure, and if that measurement is impossible to contextualize, then Netflix is providing a lot less “transparency” than it seems.

Pretty comparable, all in all, but that’s where the similarities end. Love Is Blind, of course, cost a fraction of The Crown, which has received 63 Emmy nominations. Is the former better for retention? Could the latter be a future licensing play on HBO? Without that context, and without more accurate benchmarks for success, those viewership figures provide minimal leverage to the creative teams behind them.

Netflix’s own self-reported figures don’t supply much useful context, either. While Netflix does provide individual creatives with domestic and international viewership data, its Top 10 ranking isn’t granular enough for writers or agents to create more useful benchmarks. Total hours viewed within those lists only include total global viewing. Regional lists show top trending titles, but without any viewership number.

It’s also unclear how to compare the success of a series on one platform (Netflix) versus another (Max), and what that means for each platform’s business. The WGA deal sets a threshold for series to pass from a pure viewership standpoint, but it won’t be clear if those are entirely new customers who are now paying monthly fees or already engaged customers who aren’t contributing new payments but are sticking around for the series. That inherent value in each type of customer behavior, and the underlying economics for a subscription based product, will become even more important as more companies start prioritizing revenue over exclusivity.

Again, during the linear TV era, it was simple to evaluate the success of shows using a single viewership metric, since customers paid one price for a cable bundle including nearly every channel. Today, as viewership is divided across rival platforms, each with different price points and strategies, it’s impossible to rely on just one (or even two or three) success metrics. Yes, the WGA agreement on data transparency is a step in the right direction. But it’s also important to acknowledge that it’s just a first step—and the next ones will be extremely difficult, too. The studios aren’t going to give everything away, especially when so many titles that were ordered during a period of unsustainable investment in original content aren’t performing at the level that they hoped.

If writers want more leverage to negotiate better contracts, and investors want more insight into how individual streamers are actually performing, we need new data points to create proper benchmarking across different genres, budgets, and audiences. Only then will the entire Hollywood ecosystem—writers, actors, agents, executives, analysts, etcetera—be able to talk to each other about what’s actually working, and who’s really watching on the other side of the screen. In the meantime, the data may just provide more fodder to argue about.