Generative AIs Narrow the Taste Gap

AI that writes like the average English speaker is fine. But AI that writes like you is magical.

When Nathan was building the earliest versions of Lex, Every’s AI-powered writing tool, we wanted it to eventually be able to write in the voice of whoever was using it. That seemed doable, but like it would also require lots of fine-tuning to get it to work. 

As we played with Lex we noticed something strange: it could already write in anyone’s voice—no training required.

To be sure, it’s not perfect. Depending on your writing style it might be better or worse at doing completions. But if you feed a Zen koan into Lex, it will output a koan. And if you feed Twitter hustle porn into Lex, it will output Twitter hustle porn.

This is a big part of Lex’s “wow” moment. It’s a lot like looking in a mirror for the first time. You’re seeing something familiar but also totally unexpected. 

There’s a strangeness to seeing a machine write something that you could have said or could have thought but haven’t yet. Even stranger is the idea that the machine doing it doesn’t know anything about who you are or what you think except a few lines of input text…yet it seems to be able to write as you better than any professional ghostwriter could.

I want to talk about how this is even possible, and also what its implications are. Technology is redrawing the lines around what it means to be a writer—who writers are and what they do. It’s also narrowing the taste gap—the gap between what you hope to make and what your skills allow for—by making it easier for writers to write great stuff without years of trial and error.

The coming shift is going to be profound.

How to sound like me

Here’s a very high-level explanation for how GPT-3 does text completion:

GPT-3 looks at the text that came before the point where you want completion and predicts what words are most likely to come next. In order to do this, it uses a statistical model to learn the associations between words and word sequences. The model is trained on lots and lots of source data (basically everything on the entire internet) to predict what comes next from what comes before.

How does this apply to me?

It might seem like an intractable task to figure out how to write sentences that sound like me. After all, after any sentence I write there’s an infinite number of sentences that could follow it.

But Lex shows that if you think about this problem probabilistically, it’s not as impossible as it seems. 

To make this easier to think about, you might think of the set of possible sentences as an infinite space. Any particular point in the space represents a sentence that could follow the one I just wrote. 

Given that the space is infinite, in principle, it should be hard for a machine to find a sentence in that space that’s close to the one I would’ve written on my own. But in practice, it seems that I (and everyone else) like to play around in a comparatively small corner of all of the possible sentences that we could write. 

For example, the sentences that follow from whatever I’m writing are very likely to follow the rules of English grammar (with some minor exceptions 😉.) That narrows down the possibility space by a lot. But technically, its size is still quite large.

There’re more constraints, though, than meets the eye. 

If grammar deals with the rules of sentences, Lex and GPT-3 make it clear that when we write we’re following a great number of rules that aren’t limited to things like syntax. In fact, there’s a sort of tribal grammar to what we write—the way we say things, and even what we say and think—that constrains us much more deeply than any of us care to admit. 

Lex has an easy time mimicking me because it has read everything that has influenced the way I write. (It’s probably also read my writing.) It hasn’t only picked up on style and tone—it’s also imbibed the ideas that I spend a lot of time immersed in. It knows I’m more likely to insert a reference to “network effects” into my writing than I am to reference “negative capability” because based on just a few sentences of input text it can tell I think more about startups than I do about Romance-era poets.

I use a tribal grammar that’s built on all of the people and ideas I’ve read and interacted with in my life—and compared to the set of all possible people, it’s a small list. The people who read my writing are part of the same tribe with the same set of fundamental paradigms.

Because of all of this, what I’m likely to say and think next is a statistically tractable problem. We don’t admit this because, culturally, we put a huge premium on freedom of thought and of speech. 

When it comes to what we say and think we imagine ourselves as explorers, free to roam an infinite expanse of space. But in reality we're a lot more like farmers in the 1800s: most of us never leave the towns where we reside. Some of us do move, but it's slow and it takes a while.

Lex, language, and free will

It might seem depressing or scary or infuriating to know that we operate by a sort of tribal grammar. It feels so…deterministic. We thought we had free will, but really we’ve all been NPCs this whole time. 

---

Want to read the rest of this article and skip the 2ok+ waitlist for Lex? Become an Every paid subscriber today...