How The Times is using AI to model synthetic focus groups from human audiences

“I got very nervous when you used the words ‘synthetic audiences,’” said Tracy Yaverbaun, gm of The Times and Sunday Times, on stage for a live recording of the Digiday Podcast during last week’s Digiday Publishing Summit Europe. The publication’s preferred wording is “synthetic research.”

Synthetic audiences or synthetic research, either phrase refers to the use of AI to create an artificial – i.e. synthetic – audience modeled on an actual human audience so that a company can use that AI-generated audience as a sort of focus group, i.e. for research purposes.

Ad agencies such as Dentsu have developed synthetic audiences to inform their media planning processes. And The Times is similarly using synthetic audiences research to guide its editorial product plans, among other ends.

“We think about synthetic research as a way to get to market quicker, get an answer quicker, take the guesswork out of what we’re planning to do or thinking of doing,” Yaverbaun said.

The Times began its synthetic audience research work roughly nine months ago. Working with a company called Electric Twin, the publisher has taken a reader panel from its database of 642,000 subscribers and created an AI-generated simulation, as well as an AI-generated audience panel modeled off the broader British newsreading population. 

“We’ve got our own data that we [use to] poll our own subscribers, and then they made a clone of that. So we ask them questions together,” Yaverbaun said. 

The Times queried the synthetic audience research panel when it was deciding on the name for a new business podcast that was eventually dubbed “The Business.”

“[The podcast’s name] sounds pretty obvious, but we gave [the synthetic audience research panel] about four different names, and that was the one. It came up pretty high with our reader panel, but it came really, really high of the demographic that we were really trying to target for this particular podcast,’ said Yaverbaun.

Here are a few highlights from the conversation, which have been edited for length and clarity.

The accuracy of synthetic audiences research

The first thing you think about synthetic research is, Can I trust it? How accurate is it? Pretty accurate, as it turns out. When evaluating the accuracy of the data set, we used a 10% holdout group on our own audience that we knew that we could trust. There was [an accuracy score] of .918, which is about 92% accurate. General research is only 93% accurate.

The makeup of synthetic audiences research

A lot of these synthetic research models, they use data and LLMs. And the difference between Electric Twin then for us was that layer of behavioral data they’ve also added in. The data they use is basically market research, publicly available customer data, government data that’s available to everyone. Then there’s models that sit behind that; so the models behind ChatGPT, all of the models. But then what they do is they overlay all of the social science behavioral data on top of that as well, and that goes into the model.

The training of synthetic audiences research

[Electric Twin was] embedded in our teams for quite a few months. Their team sat with our data teams, our insights teams. They looked at the panel; they looked at all of our data. They wanted to make sure that they could replicate the model in the best way possible. And then we trained side by side. So we would ask a question; they would ask a question.

Another use case for synthetic audiences research

One of the questions we asked the reader panel was: If you could have one new way to experience The Times that would make you more engaged, what would that be? What would drive deeper engagement? The difference in our reader panel research and what the newsreading population of the U.K. would look like our prospects were very, very different. Our loyal readers are very skeptical about AI. The prospect audiences wanted different types of features: summaries, AI explainers, more video.