Transcript: What Do LLMs Tell Us About the Nature of Language—And Ourselves?

The transcript of AI & I with Robin Sloan is below for paying subscribers.

Timestamps

1. Introduction: 00:00:53
2. A primer on Robin’s new book, Moonbound: 00:02:47
3. Robin’s experiments with AI, dating back to 2016: 00:04:05
4. What Robin finds fascinating about LLMs and their mechanics: 00:08:39
5. Can LLMs write truly great fiction?: 00:14:09
6. The stories built into modern LLMs: 00:27:19
7. What Robin believes to be the central question of the human race: 00:30:50
8. Are LLMs “beings” of some kind?: 00:36:38
9. What Robin finds interesting about the concept of “I”: 00:42:26
10. Robin’s pet theory about the interplay between LLMs, dreams, and books: 00:49:40

Transcript

Dan Shipper (00:00:53)

Hey, Robin. Welcome to the show.

Robin Sloan (00:00:55)

Hey, Dan! Good to be here.

Dan Shipper (00:00:57)

Good to have you. So, for people who don't know, you are the best-selling author of Mr. Penumbra's 24-Hour Bookstore, Sourdough, and, now, your latest book, Moonbound, which comes out next week. It will be out by the time this show comes out. I've read the entire thing—it's really good. I'm so excited to have you to talk about it.

Robin Sloan (00:01:18)

I'm excited to be here. Yeah, you should add to my bio, Robin is also a Dan Shipper superfan and listener and reader. So it's great to be here.

Dan Shipper (00:01:25)

I love to hear that. So, we did an interview four years ago or three years ago, which is honestly one of my favorite interviews I've ever done. And what I have to tell you, so the interview is called “Tasting Notes with Robin Sloan,” which I thought was the coolest title ever and one of the things that came out of that interview is just this sort of vast set of notes that you keep. Basically anything that comes into your periphery that has what you said, like a specific taste that has that feel for you've saved. And you said the feeling is ineffable. I got to tell you, I have an Apple note called “My Ineffable List.” That is, I've been keeping this up for four years. It's got so much stuff in it. And it's because of you. So, yeah, it really changed my life.

Robin Sloan (00:02:15)

That’s awesome. The great thing about keeping notes so assiduously and trying to cultivate a sense for that stuff that just appeals to you in that hard-to-describe way is you're essentially writing the perfect blog for yourself. And so you find going back through it, you're like, wow, amazing, all of these things are incredibly interesting to me and now I'd like to pursue them and follow up. And this is great. It's great. It's a strange media property with exactly one very, very enthusiastic—. 

Dan Shipper (00:02:47)

Incredible. So what I want to talk to you about today is your new book, Moonbound. And I'll just try to summarize it a bit for people or without spoiling it, obviously, just so we have a little bit of context. But basically it's this mashup. It's got notes like sci-fi and fantasy. It's got some Ursula K. Le Guin in it. It's got King Arthur. It's got Studio Ghibli. It's all this got all this stuff. It's really cool. It's about a boy. It's 11,000 years in the future and he's run into this downed spaceship, but also there are knights and swords and there's futuristic technology. So you're like what's going on here? And then it all sort of unravels in this really interesting way and there's a lot in it that sort of reminds me of language models. And I think you were inspired by thinking about language models and high-dimensional spaces and all that kind of stuff. So that's why we're talking about it today. And the place I wanted to start is I think you got started working on this book because you wanted to write with AI. So you're going to write it with AI and then you didn't do that, but you ended up writing it about AI. So, tell us about that process for you.

Robin Sloan (00:04:05)

Yeah such a great example—and there's no shortage of these. The best laid plans—you never go down the path you think you're going to go down. But sometimes that's all for the best. Yeah, I can probably frame it up best by rewinding in time quite a ways, actually, it's kind of shocking when you realize. I started tinkering with this stuff—and by this stuff, I mean, language models, early forms of them—way back in 2016 or 2017. If you go back and look at the code samples from that time and from people's excited blog posts you can see that it was an incredible ferment, a long way off from the stuff we have access to today. I mean, this was a moment when people were feeding in, if you can imagine, the entire corpus of Shakespeare and generating cruddy, fake Shakespeare, which is no longer impressive. At the time it was impressive and it was new and exciting. And so I got plugged into this stuff way back then. 

And I have to say that actually one of the really appealing things about that era of models, that generation, that kind of point the technology was there for me, as a writer, was twofold: One, the output actually was really weird. It wasn't as fluent as a Claude or a GPT-4. It was pretty messed up. But for aesthetic purposes, almost for poetic purposes, that was really interesting. The idea that it was written in kind of a broken, weird, inhuman way that I am your human would never imagine to write. So that was one thing that was very interesting. And the other thing that was interesting was the fact that back then the scale of everything was so much smaller that one of your big considerations could be, what will I train this on? Now, to ask that question, you have to be a multi-jillion dollar lab or big tech company to be like, oh, well, I guess I'll download the entire internet and make five copies and get started. But back then, some of my earliest experiments involved downloading huge swaths of classic public domain fantasy and science fiction and training these little models up on that. And they'll put to me was really interesting. Again, it was kind of screwed up and kind of weird, but evocative and surprising and all these other things.

That's a lot of backstory, but that's just all to say that that led me down this path of experimentation. I made, I don't know if it was actually the first one in the world, but it was for sure one of a handful of the first text editors where you could write alongside an AI model. And again, these are my cruddy primitive AI models, but I could start a sentence on a dark and stormy night dot, dot, dot, and hit tab. And this model would spin up and complete the sentence for me. I got really excited by this as I think a lot of people enmeshed in this stuff around that time—2016, 2017, 2018. And yeah, my goal, I actually didn't even have the idea for. The story that would become Moonbound at that time in a way, maybe this is a bit of a warning sign.

I was starting with the process. I was starting with this idea that I wanted to both develop and then use these tools to write in a new way. Anyway, long story short, I worked on that for quite a few years and in the end, even as the technology advanced so significantly, it got so much more capable, so much more fluent, I discovered two things. One was that that experience of writing fiction, writing creatively with the machine, was for me, actually not very much fun. And certainly didn't produce results at the level that I needed it to produce. It frankly wasn't up to spec. So that was one discovery. The other discovery, though, is that the actual machines, the stuff, the code, not their output, but the language models themselves and the math that made them go and the code that kind of wove them all together was super duper interesting. And I actually just found myself almost compulsively tinkering rather than writing kind of procrastinating because that work was so interesting. So, what happens: Moonbound out now does not have a scrap of AI-written code, AI-written text in it, but it's packed full of these ideas and actually some of these feelings that I gleaned from all that time spent with this technology.

Dan Shipper (00:08:33)

That's really interesting. I have so many different things I want to ask you about. But the thing that's coming into my mind is, yeah, tell us about the ideas. Tell us about the feelings. What did being close to these models mean to you? And how did it change how you think about the world?

Robin Sloan (00:08:50)

Yeah there was one early on, and it's kind of off to one side from the main line of development of the language models, and there was this phenomenal project that some researchers at Stanford did. They've since gone off to professorships at Columbia and other places. But they took sentences like a huge corpus—again, huge for the time—of sentences. And so it wasn't a language model. It wasn't trained on that generation task of, sort of, okay, I say this, you say that. Instead, they just wanted to take those sentences and pack them into an embedding space. And I think a lot of people who use these models maybe know what that is. If not, we can address that separately, but suffice it to say, they wanted to take these sentences and pack them into the space and set it up in such a way so that you could actually move between the sentences in a way that was sort of sensible. So you could start with a sentence, like, you know, it was hot outside. And then you'd have another sentence that was like, the dog barked at the moon, or something like that. And you could actually crossfade between the sentences, and what the operation meant is you're literally moving through this high-dimensional space with about a thousand coordinates, in their case. And I just remember, for me, I, again, those spaces and those long lists of coordinates, they're part of the generative language models too. But in this particular case, imagining all these different sentences, kind of floating in this amorphous cloud that had some meaning to it. The idea that language could get mapped into math in this way was just so freaking cool. And I just found it so, again, almost in a poetic sense, evocative and provocative. And I just wanted to keep thinking about that.

Dan Shipper (00:10:34)

Yeah. And, so, for people who aren't fully up on embedding spaces—

Robin Sloan (00:10:36)

Yeah, I know. We can dive into that too. We can do a little tutorial maybe.

Dan Shipper (00:10:38)

Basically what you're talking about is we've figured out ways to, given a set of sentences or a sequence of text, to basically map that text on a map where things that are closer together are closer in meaning. But instead of being a two-dimensional map, it's many dimensional, which is very hard to think about, but where each dimension has a certain kind of meaning. So in your book, there's actually people who are swimming through a many-dimensional space, which I really love, it's very cool. And one of the dimensions is bagel-ness so like the more you go in the bagel-ness direction, the more bagel-y it is, the more bagel-y it is. And this is in the book, but it's also real. Anthropic has this whole new feature paper where they pulled out all the features inside language models. And they have the Golden Gate Bridge feature and you can tune it to always activate the Golden Gate Bridge, which I really love. I think that's really cool.

Robin Sloan (00:11:46)

Yeah, to give you a sense of the real vintage here, sometimes I feel like I just got into this stuff too early and I basically did all of my experience and I completely burned out and then promptly the field exploded into stunning success, but it might've been 2017, 2018, where in this same little office where I'm talking to you from, a friend of mine and I, we took that sentence space that I created and I plowed all these science fiction sentences into the space and I basically created a whole list of these runs, in which for each dimension, not all of them, not all thousand, but I think I picked about 60, all the coordinates were held constant except for one. And we just moved through that dimension printing out sentences as we went. And it looked like experimental poetry. There are these printouts of these just absolutely wonky sentences. Some of them were gibberish. Actually, some of them didn't actually make sense grammatically. But the idea was we were going to identify what the dimensions meant. I was looking for it. Bagel-ness and irony and or sincerity or descriptiveness or whatever, it did not work, but we did identify that a couple dimensions obviously had to do with sentence length. Kind of the most basic, boring thing you can imagine. But the rest, we were looking at these lists of these transformations and we kind of went, I don't know, man. So I found it. I do actually find it a little bit reassuring or satisfying that it took until now it took until 2024 for the leading AI labs to find ways to interpret these features in these dimensions. Because I couldn't do it, but I tried.

Dan Shipper (00:13:20)

Yeah, no, I've seen some, I've seen people have demos. Like my friend Linus—Linus Lee, who I interviewed on this show, has one of those where you can sort of scrub a sentence from being really concise to being really long as one dimension. But then he has other ones that are weirder, being more about space versus less about space. And that kind of stuff is really cool. I'm sort of interested in this. It seems like you were so excited about it. You got so into it. Seems like you burned out a little bit, but now there's this resurgence and, what do you think was not quite working for you about it? Because, I think there's a lot of reasons why people end up writing off this technology. And a lot of them are not super curious about it, but you are super curious about it. But you're also like, I'm not really like using it that much. So, tell me about that.

Robin Sloan (00:14:18)

Yeah, and specifically in the creative writing context, right? We can restrict our focus to that because that's where I was most focused and a bit kind of obsessed, lightly obsessed, for several years and also where I sort of, at the end, had to close the book and say, this is not going to work for me. And I would say that it had to do—you know, I'm thinking even beyond my own tinkering to to a project I worked on maybe a couple of years ago now, with a great Google model, not as high end as their latest, but I mean, it was a very capable model called LaMDA and they, to their credit, had done this and it's an amazing work to wire it up into like this writing interface—super cool, fluent, I mean, it just made it so potentially interesting and powerful to be able to kind of work with text and have the AI do these completions and you could kind of guide it in all these different ways. And so they had signed up several writers to test it out and they were going to publish their short stories—whatever emerged from this engagement—in a little online anthology. And this, for me, kind of was the test and the real kind of turning point because I was like, all right, maybe my stuff was all crap, this is not good enough, actually. And now we've got these super capable models, this amazing interface. Let's try this for real. And what I discovered is that while the language model trick of sort of fitting into a style and a mode and parroting back, oh, it's a murder mystery. Oh, it's high fantasy. Oh, it's a business memo, whatever. It's really impressive. And especially when you kind of squint and say, oh, wow, I can't believe it can do that. It's really impressive. When you are working at the level of, I would like to think of fairly high end fictional composition, you see that it's always close, but never quite exactly right. And that has to do with a kind of intention. When I'm writing something, for instance, I was writing this story. I see this as a thing—I actually don't know exactly how to say what I was doing. So, it’s in my head, and I know what it is when it comes out in the words. But the point is you can't if you can write, oh, it's a classic sci-fi pulp fantasy, it actually means that's not worth writing because you want to write something that only the work itself can describe, but even so I had this text going and I would say, okay, your turn Google AI model. And it put in something, I just was like, no, you don't get what I'm doing here it was. Obviously, it was grammatically correct. It was fluent. It was fine. But it wasn't great. And, boy, it's hard enough to make a piece of writing work and make it worth publishing when everything is great. I mean, that's not the goal. That's the beginning, that’s the starting line to make it all great. And so in the end, I just was like, I gotta do this myself. And that was interesting.

And you see, my diagnosis is that there actually is a reason for that. And that has to do with the fact that all these language models are essentially generating text from inside a distribution, right? A distribution of contents, this fuzzy cloud. I don't know what the most generic phrase in all of languages is, you know, hello there, whatever it is that's obviously the supernova hot center of this cloud. And then it goes out and out and they cover the statistical terrain. And I think the truth is good—really, really, really good writing is way out at the edge of that probability cloud, that distribution of content. And I mean, I think truly good writing actually pushes a bit beyond it. It's the stuff that expands the frontier of what we thought could be written. And that's precisely where language models are the weakest. So there you go.

Dan Shipper (00:18:05)

That's really interesting. Have you tried, I don't know, either prompt tuning something like Claude, which I found to be quite good at changing its voice to your specifications, or even -tuning some of the more frontier models of today on that science fiction corpus or anything like that?

Robin Sloan (00:18:22)

Yeah, fine-tuning is a really interesting question. I haven't yet. Perhaps I will in time, but I'll tell you, I have some reservations at this moment. And they have to do with all the other stuff that's in there. And it's a little paradoxical. Let's just say that I had a list of my favorite 30 authors. And I don't know that I would do this. There's a lot of, a lot of questions built into this. But let's just say I decided to proceed: My favorite 30 authors, I had the full text of all their stuff. And I was like, I want the ultimate voice, right? I want it to reflect all those. Now, That's not enough to train a model from scratch, as we know, that's simply not enough data. It's paltry. And so what, as you say, what you have to do is you have to fine-tune one of these incredibly capable supermodels that have been trained on everything ever written. And for me, I actually I'm quite uneasy about the knowledge that even though it's been fine-tuned on this stuff that I provided, all that other content is still lurking in its training. And it's, I mean, the wild thing to me about all these corpora—any corpus in the year 2024, by definition, it's an artifact that cannot be read by a person. It cannot be read and checked by a person. I mean, it's just at a scale that's only computational. And so even the makers, the custodians of these models, obviously they can spot check. They can write other computer programs. They could even use other AI models to sort of filter or sort or select or evaluate these huge bodies of data. But fundamentally, they don't know what's in there. And I don't know, maybe that's okay for a helpful virtual assistant. Maybe it's not. For my purposes, the idea that these are going to be thoughts and feelings and ideas that are going to come out in fiction, that not knowing really, really makes me uneasy. I don't know.

Dan Shipper (00:20:20)

One of the things I really think is interesting about your work and what you touched on with me in our last interview is that you make content, but you also think a lot about the container within which the content comes. And how much of this do you think is a problem of sort of shunting a new way to make content into an old form and how much of it is, if there was a different container for this, it would be a lot more useful.

Robin Sloan (00:20:47)

Yeah. I mean, what that makes me think of is, of course, some sort of hyperbook or living book right where you say, yeah, instead of I'm not going to use a language model to to bake out a bunch of text that I that I think is Robin-level, whatever that is, I'm gonna have it—

Dan Shipper (00:21:02)

There's a dimension for that, I bet. 

Robin Sloan (00:21:04)

Max it out! Full Robin-ness! Dial goes up to 11! And so, I mean, I think that's a really interesting thing to imagine. But I have to confess, I'm sort of stuck at an issue that I had, and I worried about this five years ago, six years ago, and I would still, if I was someone building any kind of AI powered thing including a cool hyperbook of the future, I would feel so uncomfortable putting people in front of a product or a artifact or whatever you call it, where on some fundamental level, I did not know what it was gonna say. And, to me this is gonna sound very silly or funny or naive, but I'm actually surprised that these big companies have been comfortable releasing these systems to the world with that fundamental uncertainty in front of them. And now, obviously, I know they've done a ton of work to put in these guardrails and these filters. And some people would argue that they've done too much. But for me, the truth is—I guess this just says more about me than it does about AI or tech companies or society or anything. If I got an email from someone saying, oh, hey, I spent some time with your hyperbook. And, yeah, look, it showed me the scene and isn't that kind of disturbing? And if I read it and it was disturbing, I don't know, I'd shut it down. I would not be comfortable with that. So, that's a real question to answer or dilemma to kind of worm your way through, I think.

Dan Shipper (00:22:52)