Transcript: What Do LLMs Tell Us About the Nature of Language—And Ourselves?

The transcript of AI & I with Robin Sloan is below for paying subscribers.

Timestamps

1. Introduction: 00:00:53
2. A primer on Robin’s new book, Moonbound: 00:02:47
3. Robin’s experiments with AI, dating back to 2016: 00:04:05
4. What Robin finds fascinating about LLMs and their mechanics: 00:08:39
5. Can LLMs write truly great fiction?: 00:14:09
6. The stories built into modern LLMs: 00:27:19
7. What Robin believes to be the central question of the human race: 00:30:50
8. Are LLMs “beings” of some kind?: 00:36:38
9. What Robin finds interesting about the concept of “I”: 00:42:26
10. Robin’s pet theory about the interplay between LLMs, dreams, and books: 00:49:40

Transcript

Dan Shipper (00:00:53)

Hey, Robin. Welcome to the show.

Robin Sloan (00:00:55)

Hey, Dan! Good to be here.

Dan Shipper (00:00:57)

Good to have you. So, for people who don't know, you are the best-selling author of Mr. Penumbra's 24-Hour Bookstore, Sourdough, and, now, your latest book, Moonbound, which comes out next week. It will be out by the time this show comes out. I've read the entire thing—it's really good. I'm so excited to have you to talk about it.

Robin Sloan (00:01:18)

I'm excited to be here. Yeah, you should add to my bio, Robin is also a Dan Shipper superfan and listener and reader. So it's great to be here.

Dan Shipper (00:01:25)

I love to hear that. So, we did an interview four years ago or three years ago, which is honestly one of my favorite interviews I've ever done. And what I have to tell you, so the interview is called “Tasting Notes with Robin Sloan,” which I thought was the coolest title ever and one of the things that came out of that interview is just this sort of vast set of notes that you keep. Basically anything that comes into your periphery that has what you said, like a specific taste that has that feel for you've saved. And you said the feeling is ineffable. I got to tell you, I have an Apple note called “My Ineffable List.” That is, I've been keeping this up for four years. It's got so much stuff in it. And it's because of you. So, yeah, it really changed my life.

Robin Sloan (00:02:15)

That’s awesome. The great thing about keeping notes so assiduously and trying to cultivate a sense for that stuff that just appeals to you in that hard-to-describe way is you're essentially writing the perfect blog for yourself. And so you find going back through it, you're like, wow, amazing, all of these things are incredibly interesting to me and now I'd like to pursue them and follow up. And this is great. It's great. It's a strange media property with exactly one very, very enthusiastic—. 

Dan Shipper (00:02:47)

Incredible. So what I want to talk to you about today is your new book, Moonbound. And I'll just try to summarize it a bit for people or without spoiling it, obviously, just so we have a little bit of context. But basically it's this mashup. It's got notes like sci-fi and fantasy. It's got some Ursula K. Le Guin in it. It's got King Arthur. It's got Studio Ghibli. It's all this got all this stuff. It's really cool. It's about a boy. It's 11,000 years in the future and he's run into this downed spaceship, but also there are knights and swords and there's futuristic technology. So you're like what's going on here? And then it all sort of unravels in this really interesting way and there's a lot in it that sort of reminds me of language models. And I think you were inspired by thinking about language models and high-dimensional spaces and all that kind of stuff. So that's why we're talking about it today. And the place I wanted to start is I think you got started working on this book because you wanted to write with AI. So you're going to write it with AI and then you didn't do that, but you ended up writing it about AI. So, tell us about that process for you.

Robin Sloan (00:04:05)

Yeah such a great example—and there's no shortage of these. The best laid plans—you never go down the path you think you're going to go down. But sometimes that's all for the best. Yeah, I can probably frame it up best by rewinding in time quite a ways, actually, it's kind of shocking when you realize. I started tinkering with this stuff—and by this stuff, I mean, language models, early forms of them—way back in 2016 or 2017. If you go back and look at the code samples from that time and from people's excited blog posts you can see that it was an incredible ferment, a long way off from the stuff we have access to today. I mean, this was a moment when people were feeding in, if you can imagine, the entire corpus of Shakespeare and generating cruddy, fake Shakespeare, which is no longer impressive. At the time it was impressive and it was new and exciting. And so I got plugged into this stuff way back then. 

And I have to say that actually one of the really appealing things about that era of models, that generation, that kind of point the technology was there for me, as a writer, was twofold: One, the output actually was really weird. It wasn't as fluent as a Claude or a GPT-4. It was pretty messed up. But for aesthetic purposes, almost for poetic purposes, that was really interesting. The idea that it was written in kind of a broken, weird, inhuman way that I am your human would never imagine to write. So that was one thing that was very interesting. And the other thing that was interesting was the fact that back then the scale of everything was so much smaller that one of your big considerations could be, what will I train this on? Now, to ask that question, you have to be a multi-jillion dollar lab or big tech company to be like, oh, well, I guess I'll download the entire internet and make five copies and get started. But back then, some of my earliest experiments involved downloading huge swaths of classic public domain fantasy and science fiction and training these little models up on that. And they'll put to me was really interesting. Again, it was kind of screwed up and kind of weird, but evocative and surprising and all these other things.

That's a lot of backstory, but that's just all to say that that led me down this path of experimentation. I made, I don't know if it was actually the first one in the world, but it was for sure one of a handful of the first text editors where you could write alongside an AI model. And again, these are my cruddy primitive AI models, but I could start a sentence on a dark and stormy night dot, dot, dot, and hit tab. And this model would spin up and complete the sentence for me. I got really excited by this as I think a lot of people enmeshed in this stuff around that time—2016, 2017, 2018. And yeah, my goal, I actually didn't even have the idea for. The story that would become Moonbound at that time in a way, maybe this is a bit of a warning sign.

I was starting with the process. I was starting with this idea that I wanted to both develop and then use these tools to write in a new way. Anyway, long story short, I worked on that for quite a few years and in the end, even as the technology advanced so significantly, it got so much more capable, so much more fluent, I discovered two things. One was that that experience of writing fiction, writing creatively with the machine, was for me, actually not very much fun. And certainly didn't produce results at the level that I needed it to produce. It frankly wasn't up to spec. So that was one discovery. The other discovery, though, is that the actual machines, the stuff, the code, not their output, but the language models themselves and the math that made them go and the code that kind of wove them all together was super duper interesting. And I actually just found myself almost compulsively tinkering rather than writing kind of procrastinating because that work was so interesting. So, what happens: Moonbound out now does not have a scrap of AI-written code, AI-written text in it, but it's packed full of these ideas and actually some of these feelings that I gleaned from all that time spent with this technology.

Dan Shipper (00:08:33)

That's really interesting. I have so many different things I want to ask you about. But the thing that's coming into my mind is, yeah, tell us about the ideas. Tell us about the feelings. What did being close to these models mean to you? And how did it change how you think about the world?

Robin Sloan (00:08:50)

Yeah there was one early on, and it's kind of off to one side from the main line of development of the language models, and there was this phenomenal project that some researchers at Stanford did. They've since gone off to professorships at Columbia and other places. But they took sentences like a huge corpus—again, huge for the time—of sentences. And so it wasn't a language model. It wasn't trained on that generation task of, sort of, okay, I say this, you say that. Instead, they just wanted to take those sentences and pack them into an embedding space. And I think a lot of people who use these models maybe know what that is. If not, we can address that separately, but suffice it to say, they wanted to take these sentences and pack them into the space and set it up in such a way so that you could actually move between the sentences in a way that was sort of sensible. So you could start with a sentence, like, you know, it was hot outside. And then you'd have another sentence that was like, the dog barked at the moon, or something like that. And you could actually crossfade between the sentences, and what the operation meant is you're literally moving through this high-dimensional space with about a thousand coordinates, in their case. And I just remember, for me, I, again, those spaces and those long lists of coordinates, they're part of the generative language models too. But in this particular case, imagining all these different sentences, kind of floating in this amorphous cloud that had some meaning to it. The idea that language could get mapped into math in this way was just so freaking cool. And I just found it so, again, almost in a poetic sense, evocative and provocative. And I just wanted to keep thinking about that.

Dan Shipper (00:10:34)

Yeah. And, so, for people who aren't fully up on embedding spaces—

Robin Sloan (00:10:36)

Yeah, I know. We can dive into that too. We can do a little tutorial maybe.

Dan Shipper (00:10:38)

Basically what you're talking about is we've figured out ways to, given a set of sentences or a sequence of text, to basically map that text on a map where things that are closer together are closer in meaning. But instead of being a two-dimensional map, it's many dimensional, which is very hard to think about, but where each dimension has a certain kind of meaning. So in your book, there's actually people who are swimming through a many-dimensional space, which I really love, it's very cool. And one of the dimensions is bagel-ness so like the more you go in the bagel-ness direction, the more bagel-y it is, the more bagel-y it is. And this is in the book, but it's also real. Anthropic has this whole new feature paper where they pulled out all the features inside language models. And they have the Golden Gate Bridge feature and you can tune it to always activate the Golden Gate Bridge, which I really love. I think that's really cool.

Robin Sloan (00:11:46)

Yeah, to give you a sense of the real vintage here, sometimes I feel like I just got into this stuff too early and I basically did all of my experience and I completely burned out and then promptly the field exploded into stunning success, but it might've been 2017, 2018, where in this same little office where I'm talking to you from, a friend of mine and I, we took that sentence space that I created and I plowed all these science fiction sentences into the space and I basically created a whole list of these runs, in which for each dimension, not all of them, not all thousand, but I think I picked about 60, all the coordinates were held constant except for one. And we just moved through that dimension printing out sentences as we went. And it looked like experimental poetry. There are these printouts of these just absolutely wonky sentences. Some of them were gibberish. Actually, some of them didn't actually make sense grammatically. But the idea was we were going to identify what the dimensions meant. I was looking for it. Bagel-ness and irony and or sincerity or descriptiveness or whatever, it did not work, but we did identify that a couple dimensions obviously had to do with sentence length. Kind of the most basic, boring thing you can imagine. But the rest, we were looking at these lists of these transformations and we kind of went, I don't know, man. So I found it. I do actually find it a little bit reassuring or satisfying that it took until now it took until 2024 for the leading AI labs to find ways to interpret these features in these dimensions. Because I couldn't do it, but I tried.

Dan Shipper (00:13:20)

Yeah, no, I've seen some, I've seen people have demos. Like my friend Linus—Linus Lee, who I interviewed on this show, has one of those where you can sort of scrub a sentence from being really concise to being really long as one dimension. But then he has other ones that are weirder, being more about space versus less about space. And that kind of stuff is really cool. I'm sort of interested in this. It seems like you were so excited about it. You got so into it. Seems like you burned out a little bit, but now there's this resurgence and, what do you think was not quite working for you about it? Because, I think there's a lot of reasons why people end up writing off this technology. And a lot of them are not super curious about it, but you are super curious about it. But you're also like, I'm not really like using it that much. So, tell me about that.

Robin Sloan (00:14:18)

Yeah, and specifically in the creative writing context, right? We can restrict our focus to that because that's where I was most focused and a bit kind of obsessed, lightly obsessed, for several years and also where I sort of, at the end, had to close the book and say, this is not going to work for me. And I would say that it had to do—you know, I'm thinking even beyond my own tinkering to to a project I worked on maybe a couple of years ago now, with a great Google model, not as high end as their latest, but I mean, it was a very capable model called LaMDA and they, to their credit, had done this and it's an amazing work to wire it up into like this writing interface—super cool, fluent, I mean, it just made it so potentially interesting and powerful to be able to kind of work with text and have the AI do these completions and you could kind of guide it in all these different ways. And so they had signed up several writers to test it out and they were going to publish their short stories—whatever emerged from this engagement—in a little online anthology. And this, for me, kind of was the test and the real kind of turning point because I was like, all right, maybe my stuff was all crap, this is not good enough, actually. And now we've got these super capable models, this amazing interface. Let's try this for real. And what I discovered is that while the language model trick of sort of fitting into a style and a mode and parroting back, oh, it's a murder mystery. Oh, it's high fantasy. Oh, it's a business memo, whatever. It's really impressive. And especially when you kind of squint and say, oh, wow, I can't believe it can do that. It's really impressive. When you are working at the level of, I would like to think of fairly high end fictional composition, you see that it's always close, but never quite exactly right. And that has to do with a kind of intention. When I'm writing something, for instance, I was writing this story. I see this as a thing—I actually don't know exactly how to say what I was doing. So, it’s in my head, and I know what it is when it comes out in the words. But the point is you can't if you can write, oh, it's a classic sci-fi pulp fantasy, it actually means that's not worth writing because you want to write something that only the work itself can describe, but even so I had this text going and I would say, okay, your turn Google AI model. And it put in something, I just was like, no, you don't get what I'm doing here it was. Obviously, it was grammatically correct. It was fluent. It was fine. But it wasn't great. And, boy, it's hard enough to make a piece of writing work and make it worth publishing when everything is great. I mean, that's not the goal. That's the beginning, that’s the starting line to make it all great. And so in the end, I just was like, I gotta do this myself. And that was interesting.

And you see, my diagnosis is that there actually is a reason for that. And that has to do with the fact that all these language models are essentially generating text from inside a distribution, right? A distribution of contents, this fuzzy cloud. I don't know what the most generic phrase in all of languages is, you know, hello there, whatever it is that's obviously the supernova hot center of this cloud. And then it goes out and out and they cover the statistical terrain. And I think the truth is good—really, really, really good writing is way out at the edge of that probability cloud, that distribution of content. And I mean, I think truly good writing actually pushes a bit beyond it. It's the stuff that expands the frontier of what we thought could be written. And that's precisely where language models are the weakest. So there you go.

Dan Shipper (00:18:05)

That's really interesting. Have you tried, I don't know, either prompt tuning something like Claude, which I found to be quite good at changing its voice to your specifications, or even -tuning some of the more frontier models of today on that science fiction corpus or anything like that?

Robin Sloan (00:18:22)

Yeah, fine-tuning is a really interesting question. I haven't yet. Perhaps I will in time, but I'll tell you, I have some reservations at this moment. And they have to do with all the other stuff that's in there. And it's a little paradoxical. Let's just say that I had a list of my favorite 30 authors. And I don't know that I would do this. There's a lot of, a lot of questions built into this. But let's just say I decided to proceed: My favorite 30 authors, I had the full text of all their stuff. And I was like, I want the ultimate voice, right? I want it to reflect all those. Now, That's not enough to train a model from scratch, as we know, that's simply not enough data. It's paltry. And so what, as you say, what you have to do is you have to fine-tune one of these incredibly capable supermodels that have been trained on everything ever written. And for me, I actually I'm quite uneasy about the knowledge that even though it's been fine-tuned on this stuff that I provided, all that other content is still lurking in its training. And it's, I mean, the wild thing to me about all these corpora—any corpus in the year 2024, by definition, it's an artifact that cannot be read by a person. It cannot be read and checked by a person. I mean, it's just at a scale that's only computational. And so even the makers, the custodians of these models, obviously they can spot check. They can write other computer programs. They could even use other AI models to sort of filter or sort or select or evaluate these huge bodies of data. But fundamentally, they don't know what's in there. And I don't know, maybe that's okay for a helpful virtual assistant. Maybe it's not. For my purposes, the idea that these are going to be thoughts and feelings and ideas that are going to come out in fiction, that not knowing really, really makes me uneasy. I don't know.

Dan Shipper (00:20:20)

One of the things I really think is interesting about your work and what you touched on with me in our last interview is that you make content, but you also think a lot about the container within which the content comes. And how much of this do you think is a problem of sort of shunting a new way to make content into an old form and how much of it is, if there was a different container for this, it would be a lot more useful.

Robin Sloan (00:20:47)

Yeah. I mean, what that makes me think of is, of course, some sort of hyperbook or living book right where you say, yeah, instead of I'm not going to use a language model to to bake out a bunch of text that I that I think is Robin-level, whatever that is, I'm gonna have it—

Dan Shipper (00:21:02)

There's a dimension for that, I bet. 

Robin Sloan (00:21:04)

Max it out! Full Robin-ness! Dial goes up to 11! And so, I mean, I think that's a really interesting thing to imagine. But I have to confess, I'm sort of stuck at an issue that I had, and I worried about this five years ago, six years ago, and I would still, if I was someone building any kind of AI powered thing including a cool hyperbook of the future, I would feel so uncomfortable putting people in front of a product or a artifact or whatever you call it, where on some fundamental level, I did not know what it was gonna say. And, to me this is gonna sound very silly or funny or naive, but I'm actually surprised that these big companies have been comfortable releasing these systems to the world with that fundamental uncertainty in front of them. And now, obviously, I know they've done a ton of work to put in these guardrails and these filters. And some people would argue that they've done too much. But for me, the truth is—I guess this just says more about me than it does about AI or tech companies or society or anything. If I got an email from someone saying, oh, hey, I spent some time with your hyperbook. And, yeah, look, it showed me the scene and isn't that kind of disturbing? And if I read it and it was disturbing, I don't know, I'd shut it down. I would not be comfortable with that. So, that's a real question to answer or dilemma to kind of worm your way through, I think.

Dan Shipper (00:22:52)

That's really interesting. For me, that’s actually— I obviously don't want to serve disturbing stuff to people, but it is sort of exciting in a weird way. You're creating this living thing where you can't totally predict it, you know?

Robin Sloan (00:23:04)

Living thing. Yeah, I understand that. Yeah. And again, though that's actually where it gets extra interesting and, in a way, that our last two kinds of threads, I think, come back together and they tie together. If there was a system where I was able to say, listen, this is all public it's, I know what's in it. I know what went in. And now it's gonna operate in this living, organic, unpredictable way. To me, I would actually be more comfortable with that. I'd be a lot more comfortable with that. It is, I think, precisely the combination of these big— There's such amazing artifacts. Again, you could, you could spend, you could so profitably spend all your time, so much time, thinking about, dreaming about, writing science fiction stories about language models, even just as they exist today in our real world without ever— They don't have to do anything just as artifacts like what they are and what's woven into them and bound up into them—it’s just so weird! It's so interesting. But, that very thing I, again, I don't actually know. And nobody knows what's in Claude or GPT-4. And so, I don't know. I wouldn't put my name on the thing that shoots that output at unsuspecting people.

Dan Shipper (00:24:24)

You want nutrition facts—like ethically raised, organic AI.

Robin Sloan (00:24:26)

I mean, I think that's coming. Some version of that is coming. 

Dan Shipper (00:24:28)

I think so too. I think that's great. I think it's really cool. I mean, you kind of see this already with Adobe has the—we don't train on copyrighted material stuff for their image models. That's the first kind of salvo in that. I want to get into the book a little bit and I know I know for you, the idea that books are not summarizable as sort of this key thing. So I don't want to reduce it too much.

Robin Sloan (00:24:59)

I mean, no, they're not summarizable, but one must always try. That’s the paradox.

Dan Shipper (00:25:01)

Well, yeah, I think the full thing is not summarizable, but the summaries are a pointer to unsummarizable things that you might find interesting. And so you have to have the pointers because we have limited attention.

Robin Sloan (00:25:19)

Yes. Yes.

Dan Shipper (00:25:21)

So one of the ways I could summarize this book or a key idea that's woven through it is it's sort of about stories and about how stories shape our reality but also maybe how we can make decisions that author our own stories. We don't have to necessarily follow the prescriptions of the story that we're already a part of, which seems really interesting and also sort of related to the next token prediction. And I don't know if I like reading too much into it, but I'm just really curious. Tell me more about that.

Robin Sloan (00:25:57)

That's great. So first of all, I should say. The book is newly released. And so I haven't had that many conversations about it. So this is really fun, actually, to just engage with someone else who's actually read the book, wasn't working on it with me, and here are the ideas kind of bouncing around other minds. I will say that that particular association, the idea of sort of proceeding in a archetypal, or maybe even stereotypical story form and producing the next token in a sequence was not consciously in my mind, but I do think it totally checks out, I mean, that's what we're talking about, about literally these sort of patterns in templates that get used and reused in so many different ways that the language models are so good at and, I guess I want to try to find a way to say this without spoiling it, but in the course of this book, as we get to know some of these AI entities a little bit better, it turns out that those patterns and templates are of incredible, extreme importance. And, in fact, and I think this is true in our real world and in our real language models, they are kind of woven into the systems at a really foundational level only because they are repeated so often and so powerfully throughout their training corpora.

Dan Shipper (00:27:13)

And are you saying something about stories in relation to language models, or do you feel like maybe not necessarily saying it in the book, but do you feel something about stories in relation to language models?

Robin Sloan (00:27:24)

Yeah, I do. And I don't want to make claims that are too bold because the truth is, I don't know, I don't know what's going on inside these labs these days. I understand the open research world of the late 2010s a lot better. And I think this probably is changing now. Years ago when you thought about what was available in the world to scrape and collect and feed into your hungry language model, it was things with narrative. It was stuff with a story. And that doesn't have to mean fairy tales and epic quests, by the way, and myths and legends. It can just be news articles. News articles have a real appetite for cause and effect, even if something is fundamentally inexplicable or random or tragic and chaotic. A news treatment wants to say, and just because it's a human thing, they want to say, well, this probably happened because of that or maybe it happened because of this. I do think it's interesting and I don't have an answer to this. I don't really even have a theory, but I think it's interesting to speculate about how these models and their constitutions, their intuitions about language have changed with the introduction of so much code. I've heard it speculated that it was the addition of huge amounts of code to these training sets around the time of GPT-3 that actually affected their language use in a lot of ways, because code, of course, is so structured. And so if-that and consequential and maybe the thing that we imagine is reasoning in language models today comes in large part from what they've learned from code. If that's the case, I think suddenly you're like, is code a story? I don't know. 

Maybe it's a very linear story so my sense of what is in the hearts of the language models might be a little different today in the fullness of what they're learning from 2024. But I can say with confidence that in that era of Project Gutenberg and all the news stories on the web. Yes, there is a bias. There's a bias in that kind of language model towards if-then cause and effect and the rhythm of a story. And that's pretty interesting to notice because there are stories in the world, but there's a lot in the world that’s not a story either. And so a system with that bias, I think you have to be really careful.

Dan Shipper (00:29:51)

Totally. I think you're 100 percent right. I've heard that too, that even if you're training a model that you don't want it to code, you don't need it to code, toy train it on code because it just makes it smarter, which I think is—

Robin Sloan (00:30:03)

And isn’t that so interesting that, and this is a slightly further afield, but that has made me reflect on the fact that code, we think of it as a machine language. Of course, code is not for the benefit of machines. Machines have their own, and a machine has no use for Python or Ruby. That's not the language they speak—everybody knows this. They get compiled into the real language of machines. Really, code is for human benefit. It's a bridge. It's a way for us to sort of think in a more machine way. And we express that in these linguistic terms. I mean, again, you could just think about this stuff so profitably and such with such just delight for forever.

Dan Shipper (00:30:44)

There's tons of depth there. It makes me very, very excited. One of the other things that that I saw in the book, this happens early on, but it also becomes it becomes more relevant later is—and you say it outright—the central question of the human race is what happens next and when I read that, I was just like, because we've been thinking about rebranding every or repositioning it, and we were like, how should we reposition it? And we came up with what comes next as like, we answer that question. And so when I saw that, I immediately texted our editor in chief, Kate, and I was like, oh my god, look at this, look at this. So I feel like we're kind of on the same page there, but tell me what that means to you and where you came to that from.

Robin Sloan (00:31:28)

Yeah, I mean, that's really close to my heart. In fact, it's maybe the most close to my heart and, in fact, in the story, as you know, having read the book, the narrator, who is not human—the narrator is sort of a kind of a hybrid organic-technological creation who exists to chronicle human endeavors. And the chronicler says in the page that the question carved into my heart, the irresistible question is, what happens next? In that case, this character is speaking for me. That is the great question for me. I don't actually know that it is intrinsically a human universal. I think there are different humans and different human cultures that kind of care about it differently. I think there's probably some people that either don't care or they'd rather not find out. And that's fine. I don't personally attach a huge, deep value judgment to it. But I do cop to my own hunger for the question. For me, it's constitutional and that's one of the reasons I am such a huge reader of science fiction. Science fiction doesn't tell you the answer, but it suggests possible answers. And that's why I just find them so delicious to read and this is—I'm sorry to make it a little heavy, but when I personally think about death at the end of a life you know I think about things like suffering and disappointment and sickness and all that kind of stuff and I think about leaving behind things that I like and people that I love. I also think about just not learning what happens next, you know? If we lived in some sort of magical world where an angelic entity visited you in the last moments of your life and just spoiled it, just told you it's like, okay, well you did a good job. You lived a good life. Here's how it ends. Here's how it all goes down. For me, it would soften the blow. It really would.

Dan Shipper (00:33:23)

That's really interesting. I mean, I think it's definitely central. I do agree that certain cultures or individuals don't have that as their sort of the center of their lives. But, the fact that Buddhism exists and the entire whole thing is being the present and that's so hard for people, sort of says to me that there's something pretty, pretty hardwired about our brains. It's like, what's going to happen now? 

Robin Sloan (00:33:50)

What's happening next? Yeah, right. And, of course, I mean, it's interesting to think about how that plays out on different timescales, because of course I've completely gone to town in the last couple of years on another another fine podcast, one called Big Biology, all about really great, super, super interesting, rich kind of next-level next-generation biology. And one of the things they keep coming back to there is that even on the level of a cell, a dumb little cell, a little microbe, there is kind of that question of what happens in life in a way thrives, as a little, to the degree that it has a pretty good simulation of what might happen next, because that allows it to do things like plan and react to possible dangers and all the other things you can imagine life doing. So that's kind of the micro scale of what happens next. But then there's also the Dan scale of what happens next in 10 years, and then there's the sci fi scale of 10,000 years. And they're all interesting. Super interesting.

Dan Shipper (00:34:51)

And not to make everything about AI, but what strikes me is that answering that question is the thing that forces these models to bootstrap all of the knowledge that they have to be like, okay, what comes next? Answering that over and over and over again, trillions of times, which that's really interesting. I don't know if I would have thought that that would be the way that they would get smart.

Robin Sloan (00:35:15)

Yeah, that's honestly a great observation. The sense that a question is evolutionary, the dynamics of kind of natural selection evolution are so simple. And yet they make the rhinoceros and the anchovy and the Joshua tree and Robin and Dan it's like, okay, I guess it doesn't take much if you give it enough time. And likewise I don't actually think that that question is exactly evolutionary. I think it's something a little different. But I actually really think it's quite elegant to imagine that a challenge so simple could, in its fullness, require a lot of complexity and a rich picture of the world. I mean, I always take pains to say, still an incredibly incomplete picture of the world. I mean, I sometimes think, oh man, these language models. They just think it's all text out here. You poor bastards there's more than text and so slowly but surely. And I think the multimodal models are obviously interesting because they have such rich capabilities. I think they're interesting because it seems, I don't know. It seems like a better way to live. Seems like a better way to be in the world, to be able to combine different inputs and maybe find some common associations between them. The idea of just living in a world of nothing but tokens, I don't know, seems kind of like hell to me.

Dan Shipper (00:36:38)

Are you starting to think of them as beings that can live in hell? What’s your—

Robin Sloan (00:36:45)

No, not yet. But again, I should maybe try to actually do some research on this. If there are not presently, at minimum, dozens of philosophers and cognitive scientists and ethicists kind of thinking about this stuff around the world and maybe teaching introductory seminars about it and getting people together to start to have these conversations and ask these questions, then I don't know, the academy is derelict in its duty because these are really rich, interesting questions. For my part, no. I don't think these are beings yet but if you ask me what my definition of a being is, or where I draw the dotted line around beings, I don't know. So who knows?

Dan Shipper (00:37:30)

I mean, I think that the academy or philosophy question is really interesting because I studied philosophy in college and philosophy is sort of obsessed with definitions so, what is a being or whatever? And trying to answer those questions and definitions obviously are useful for certain things, but it's always struck me as kind of interesting that we've been debating definitions of things like knowledge for 2,000 years. And we know a lot of stuff. We just don't have a definition for it. And you would have thought that to build intelligent machines that had knowledge, we would have to define it before we built them. And really, we just have to ask that one question over and over, what comes next? And it just bootstraps it, which I think is an interesting comment on the project of philosophy. That, maybe these definitions are so high-dimensional that you can't get it in a philosophy book.

Robin Sloan (00:38:21)

Absolutely. Absolutely. I think that's great. I think that's totally great. Or, in a sense, again, you can go around in circles and you could have a whole successful, celebrated philosophy career, just arguing with another philosopher over a definition using language the entire time, right? You know, you're writing papers back and forth and you're denouncing each other in seminars, and maybe it turned out that the thing all along was actually in all that language. The idea that you're going to compress it down to a word or a sentence is wrong.

Actually, in that bulk of living language, and I say that because I don't really know how to assess this claim at all. But personally, almost aesthetically, I'm quite taken with the claim that what language models are language itself. Literally the technology of language given its first dose of autonomy, so it's not just something that we deploy as we want or need, but suddenly it's you rip language out of our heads in our society, kind of set it up and turn a crank on the side and it starts kind of walking around slowly and weirdly, like one of those little wind up toys, and that's again, you can argue about what's going on in language models, there is no disputing that when you talk with one, it sure seems like it's reasoning, or thinking, or deducing, or just politely answering your question. And so this argument goes, yes, that is the deduction and the reasoning and the politeness that was always inherent there in language, and we just never had this particular way of seeing it before. Again, I don't know, but I like it. I like the way it sounds.

Dan Shipper (00:40:08)

I think that's really cool. That's a beautiful metaphor. Is it the politeness that's always been in language in the sense that politeness is sort of built into our grammar and into our syntax and all that kind of stuff? Or is it the politeness that's in humans that was recorded into language?

Robin Sloan (00:40:26)

I think both. I think it's absolutely both. It's the latter, no question, but then even the former, I think it's just, again, all of human language is not TCP/IP packets. It's people and they're writing letters or trying to entertain each other with books or trying to convince other people of things very often. And convincing, it takes some tricks. It takes some patience or whatever. So yeah, I think it's just in terms of what language has always been for. Obviously, you can be very rude, and you can be very impolite. But that in itself, that's only effective ever because language understands that it's you being used between people and you want to get along.

Dan Shipper (00:41:19)

Yeah, I guess on the subject of language becoming an autonomous thing. One of the things that is present throughout the book is you're playing with this idea of what an I is—and I, as in letter I. I like me. You have the chronicler who's, I think you said it's a fungus, right?

Robin Sloan (00:41:38)

Yeah, a fungus onto which much technology has been layered—at great expense.

Dan Shipper (00:41:47)

Exactly. Yeah, you've got Clovis, another character that— There are many instances of Clovis all around the world, and they all are sort of connected, at least usually.

Robin Sloan (00:41:59)

And all have the same—rather than a sort of Borg hive mind, or the ants are all part of the same colony, it actually is a Clovis, this wandering robot in all their instances is the same person and the same personality, which maybe that sounds like a subtle distinction, but I think it's actually a pretty important one.

Dan Shipper (00:42:20)

So you're playing around with all that all the time. What is it about that keeps capturing you?

Robin Sloan (00:42:26)

Well, you spend any time with writing— And I would say, especially writing it and then also reading across languages, and I don't mean natively, just in translation, and and reading about the translations and the process, and you learn that you don't even need science fiction for this, just within the scope of human language as it exists here and now today, there are so many different ways of thinking about I and the subject, the subject of anything. One very classic example is that in the Japanese language, there's not just I, as there is in English, there's a handful of different I’s, all of them with really interesting and narratively useful meanings. And so Japanese translators, people who bring books from Japanese into English, they all complain about this because they're like, well, it's impossible. There's no equivalent. You just can't. We don't have other I’s we can slot in. And so fundamentally, this shade of meaning, which is, again, really important, can be really, really useful and give something— A little spin on the ball is essentially lost and they try to find other ways to thread it in and kind of create that feeling and I love that. And I just think even now you think about humans and how we live in the world. I think already the I—the singular I—is quite complicated. I think it has been complicated in the last 10 or 20 years, the era of the internet, because you know, well, here I am, I am standing in a room and at a certain time in human history, that meant I was just in this room and I could only talk to people in this room. But here I am talking to you, seeing your face. I am somewhere else in a very meaningful way and we just still say I, and I just feel like it's more complicated than that now our presences and our attention and our sense of where and how we can act in the world is already sort of spreading out. And it's fun to think about how you could reflect that, capture that, and play with that in in language.

Dan Shipper (00:44:32)

I think of that too, in so many ways. There are so many, I don’t know, internal family systems, you have got multiple I’s within you. That's a sort of psychological model or just in meditation. If you get deeper into it, you start to notice that you are a bit of a different self depending on the context you're in and I think it's also why when you go through a breakup, or someone dies, or whatever that's part of why you're sad. Because the I that is usually around that person, the I that becomes around that person is not gonna get activated anymore. You’re losing part of yourself.

Robin Sloan (00:45:08)

Yeah, absolutely. I can't say that I quite thought of it that way, but that's absolutely true, to which I would add, I mean, this is much less emotionally resonant, but there's also the simple I of your body systems. You've got your gut and the microbes there, which we now know have desires of their own and are blasting neurotransmitters up to be like, do this, do that. Your organs, I mean, everything, and these things all, they're not ancillary. They're not accessories to the pilot of your brain. Neuroscience and biology have definitively established that it's a big interlocking committee. And so, yeah, what is the I? Does it include my gut? I think so, but yeah, it's great. It's just weird. It's weird stuff to think about.

Dan Shipper (00:45:55)

Yeah. And I think it actually also relates a lot to language models in the sense that, when you think about, embedding spaces and what they are and how language models end up predicting what comes next, basically, in order to predict the next word in a sequence, they take the end word and then they figure out which version of that word is it. So if we take the name Robin they'll just go to everything previous to Robin in that sequence and be like, which Robin are we talking about? And that could be like, there's hundreds of thousands or millions of different Robins. But 1,000 or 2,000 of which are actually just you in different senses. And I love that idea. I think that is so cool. It's like we created this dictionary of these very, very, very specific words that we all thought were one word, but we're using that same word in like a million different ways. Yeah, it's really beautiful. 

Robin Sloan (00:46:52)

And again, I mean, the assignment is so clear. Please, linguists of the world and philosophers and everybody, let's get cracking because, and I do think this is so interesting. Folks making and deploying these models, I think a lot of them are interested in these questions. They are not, however, their most urgent questions. Their most urgent questions have to do with kind of the infrastructure and scaling these things up and turning them into businesses. And of course, I think the very kind of direct safety questions are about people going to use this to scam other people or to bring down governments or whatever. And there's just so much to dig into and I really, I do hope— Yeah, you've kind of brought this up in a few different ways, a sense in which just seeing these little mechanisms operate, it may— How do I want to say this? It raises questions about long standing projects. You kind of go, well, ah, maybe I didn't need that after all, or maybe we didn't need to do it that way. And those should be the kind of questions and realizations that kind of redirect streams of inquiry, I think.

Dan Shipper (00:48:01)

Totally. So you've been working on this book for a while. It's coming out. How does it feel?

Robin Sloan (00:48:08)

It feels good. I mean, as always, it’s a little dizzying to understand that it will soon be in other people's brains. I said earlier, I've done a couple of these. I've had a couple conversations ahead of time like this and it sounds so silly, but truly, in all cases, I have been sort of unprepared for other people to have actually thought about the book. I'm like, oh, right! Ah, you read it! And it's in your head! Oh, cool! Okay, this is weird but that's the goal. That is so the dream and the magic and I think I'm, of course, such a book nerd and such a book chauvinist, maybe that I just think, I think they do that more effectively than any other medium. I think they literally get into people's heads because they have to— You didn't just watch Moonbound on a screen, you enacted and kind of rehydrated the events and the meaning in your own language model, inside your own head, and that is something really special. So I'm just excited for it to come out. For that to happen a lot more in the coming weeks and months and hopefully years.

Dan Shipper (00:49:14)

I love it. Put me on team book chauvinism—the only kind of chauvinism I agree with.

Robin Sloan (00:49:20)

Yeah, I will say, very germane to our subject here. I keep talking about how these scholars ought to be asking more questions about their work and their projects because of what we've learned. I apply that also to writers—fiction writers—and I have been highly motivated to think about what we're doing and all this kind of stuff. I'm sort of nursing a pet theory. I don't know that it really has any evidence. So it's just going to continue to be Robin's crackpot theory. But I think that anybody who's interested in language models should really pay attention to how they dream and what dreams are and how dreams feel. Because, this is just subjective, I feel literally that the mechanism of dreaming is very similar to the mechanism of a language mode, kind of saying, okay, well, that's weird, but I'm gonna keep it going. I'm gonna do the best I can to complete this sequence. And the reason I bring it up is that I think that's also very similar to the mechanism of a novel. My pitch for novels, fundamentally, is that they are packaged dreams. Usually you don't get to choose your dreams. They just are weird or scary or surreal or boring or whatever, but you just get the dreams you get. I think this is one case where you get to sort of load a waking dream into your head. And so in that way, I think there's an interest, some not totally understood connection, kind of trilateral connection between books and dreams and language models.

Dan Shipper (00:50:48)

I love that. Do you think that that implies that the true AGI is going to have a language model as its dream generator? And we just haven't built the AGI yet?

Robin Sloan (00:51:00)

Yeah, maybe, maybe so. I don't want to say too much because we're very close to spoiler territory here, but suffice it to say that anyone who reads the book Moonbound will understand by the end how much dreams and sleep and what happens during sleep is important to me. I actually do wonder. I don't know if you know this, but there's no living things that don't sleep. I mean, literally it is a fundamental part of every biological process.

Dan Shipper (00:51:29)

Do bacteria sleep?

Robin Sloan (00:51:31)

They do. They have a phase that is a clear sleep thing. Now, are they dreaming? Well, maybe, but it's different. Certainly you go one notch up the sort of the scale of recognizable-to-us complexity and you do see things that begin to look like dreams. And so that's, I think, really fun and interesting stuff to read about and learn about. And it's clear that the experience of long term flourishing through evolution of life on earth has discovered this phase is really important. It's not negotiable and it makes me wonder. It makes me wonder if we will determine that there's some analogical thing that that AI is really ought to do either for their own long term success or health or who knows what.

Dan Shipper (00:52:21)

That's fascinating. Well, this was an incredible conversation.

Robin Sloan (00:52:25)

Very sci-fi. This is great. I'm delighted. As if the novel itself wasn't sci-fi enough, I feel like we just took it to a few further levels, which is very fun.

Dan Shipper (00:52:37)

We totally did, which I suspected was going to happen and I was very excited for. The book is Moonbound. When is it out?

Robin Sloan (00:52:44)

June 11th.

Dan Shipper (00:52:45)

June 11th. So, we'll be out by the time that this podcast is live. And thank you so much, Robin, for doing this. This is really an incredible conversation.

Robin Sloan (00:52:53)

Dan, this is great. Just a real treat. So let's do it again before too long.

Dan Shipper (00:52:56)

Sounds good.

Dan Shipper is the cofounder and CEO of Every, where he writes the Chain of Thought column and hosts the podcast AI & I. You can follow him on X at @danshipper and on LinkedIn, and Every on X at @every and on LinkedIn.