AI Focus Groups—And Soon AI Copywriters

In Michael Taylor’s latest piece for Also True for Humans, his column about working with AIs like you'd work with people, he explores what comes after the Turing test—when a human’s performance is no longer distinguishable from a human’s—in the world of advertising. He lays out the path for how AIs testing ads on other AIs will surpass the best human copywriters. Michael applies the wisdom of the crowd concept to working with AI to determine how agent-based simulations can help us make better business decisions. If you’re interested in synthetic market research and virtual audiences, join the waitlist at askrally.com.—Kate Lee

Was this newsletter forwarded to you? Sign up to get it in your inbox.

“Ninety-five percent of what marketers use agencies, strategists, and creative professionals for today will be handled by AI—and the AI will likely be able to test the creative against real or synthetic customer focus groups.”OpenAI’s Sam Altman tossed that line off in an interview few people noticed, but it lodged in my head.

As a former marketing agency owner, I know how time-consuming and expensive focus groups are. I also know that AI excels at roleplaying, so it stood to reason that I might be able to get it to act like a group of potential customers. If you could test ads against an AI audience first, I thought, it could reduce the risk of running unsuccessful ads and massively impact the advertiser's bottom line. Consumers, too, should benefit from seeing fewer irrelevant ads.

That led me to build Rally, a tool that helps marketers and entrepreneurs test their products against virtual audiences. It queries multiple AI personas simultaneously and aggregates their responses into actionable insights. At the risk of sounding immodest, it feels like a big step in solving the focus group side of the equation.

However, a limiting factor so far in setting AI loose to run a fully automated ad campaign has been on the composition side—simply put, purely AI-written ad copy isn’t great.

I believe that’s about to change. AI-generated content is at an inflection point. A recent study carried out by researchers at the University of California San Diego, GPT-4.5 showed remarkable progress on the Turing test, in which people try to distinguish a machine’s performance from a human’s. In almost 75 percent of attempts in the study, people couldn’t tell if they were talking to a human or AI.

GPT-4.5 with a persona is mistaken for human 75 percent of the time. Source: Arxiv.

When I saw this study, my first thought was how quickly this progress is going to hit the advertising industry. In no time at all ads will be superhuman, beating anything a human marketer can do given the same brief. This same thing happened with AI image models that were derided because they couldn’t properly generate hands, lulling creative professionals into a false sense of security. That issue was solved with the latest models faster than anyone was prepared for.

What kind of system do you need to build to fulfill Altman’s vision, where AI is doing 95 percent of what marketers do today? Imagine algorithms that spin up and test millions of ad variations on lifelike virtual audiences—discarding the duds and surfacing winners before a single human sees them. Let's explore the milestones on AI’s path to out‑creating the best human marketers and lay out the practical moves you can make now to stay in front of the coming shake‑up.

Become a paid subscriber to Every to unlock the rest of this piece and learn about: