Claude 3 Is The Most Human AI Yet

Every technology is described by the same words when it comes to market: faster, cheaper, smarter. With the release of Claude 3—an AI model comparable to GPT-4—its creator, Anthropic, said the typical stuff. The company published evaluations that showed its newest model was on par or slightly more powerful than its peers. It discussed how its models could be run more cheaply. 

All of this is cool, but the differences in benchmark testing or price weren’t so drastic as to warrant an essay from this column. 

The correct word, the unusual one, the word that Anthropic didn’t use and I feel is most accurate, is something weird. 

The right way to describe Claude3 is warm. 

It is the most human-feeling, creative, and naturalistic AI that I have ever interacted with. I’m aware that this is not a scientific metric. But frankly, we don’t have the right tests to understand this dimension of AI. Shoot, we might not even have the right language. 

Our team at Every includes some of the few people on the planet that have access to OpenAI’s latest public GPT models, Google’s newest Gemini 1.5 models, and Anthropic’s Claude 3. We’ve also been using LLMs for years—testing, evaluating, scrutinizing, and inventing new ways to work. 

In all that labor, we’ve never found an AI able to act as a robust, independent writing companion that can take on large portions of the creative burden—until now. Claude 3 seems to have finally done it. Anthropic’s team has made an AI that, when paired with a smart writer, results in a dramatically better creative product. 

This has profound implications. Great writing isn’t just about the prose, or about commas or sentences. Instead, great writing is deep thought made enjoyable. Until Claude 3, most AI models were good thought made presentable. Claude 3 frequently crosses the great/enjoyable rubicon. 

To determine that I had it work on the hardest pitch of all time: convincing someone to follow Jesus. 

Jesus, automated

Every Monday at 2 p.m. ET the entire brood of my in-laws jumps on a Facebook Messenger call. The purpose of our weekly chat is to connect with my youngest brother-in-law, Gage, who is on a Mormon mission to Brazil. We tell him about how his favorite sports teams are doing, he tells us about who he is teaching about Jesus that week. Many of us on the call donned the necktie-and-white-shirt combo and served missions ourselves, so we share pointers on what worked for us. 

This Monday, I had an idea. Rather than rely on our experience for guidance, we could summon the collective wisdom of mankind—i.e., we could ask the LLMs for advice on missionary work. So I pulled up Claude 3, GPT-4, and Gemini 1.5, and prompted them with a question about how to help people learning about the Church understand the long list of rules that members are asked to follow. 

First, I asked GPT-4 to help my brother-in-law teach in a more compelling way. It outputted a long list of generic advice that felt cold. 

Source: Screenshot by the author.(This went on for another 11 bullets.) Then I tried Gemini. The result was similarly impersonal writing, with comparable substance—and it took an extra 20 seconds. 

Source: Screenshot by the author.Finally I tried Claude 3. It was remarkably gentle, kind, and specific. I was astonished.

Source: Screenshot by the author.While this test was interesting, it wasn’t possible to draw general conclusions from it. The more compelling example was when we deployed it in our own products at work.