Transcript: ‘Dwarkesh Patel’s Quest to Learn Everything’

The transcript of AI & I with Dwarkesh Patel is below for paying subscribers.

Timestamps

Introduction: 00:01:44
How Dwarkesh uses LLMs to remember everything: 00:05:37
Dwarkesh’s taste in books and how he uses AI to learn from them: 00:11:50
Why it’s important to be an early adopter of technology: 00:17:58
How Dwarkesh uses Claude to understand complex concepts: 00:20:44
Dwarkesh on how you can compound your intelligence: 00:26:36
Why Dwarkesh is on a quest to know everything: 00:28:21
Dan and Dwarkesh prep for an upcoming interview: 00:39:19
How Dwarkesh uses AI for post-production of his podcast: 1:04:14
Rapid fire on AI’s biggest questions—AGI and P(doom): 1:08:51

Transcript

Dan Shipper (00:01:48)

Dwarkesh, welcome to the show.

Dwarkesh Patel (00:01:49)

Thanks for having me, Dan.

Dan Shipper (00:01:50)

I'm so excited to have you. For people who don't know you—I assume everyone knows you. But for people who don't, you do the best, honestly, the smartest interviews in AI that I've found. You have really incredible guests like Mark Zuckerberg, Demis Hassabis, Patrick Collison. You've created the go-to show for smart people to learn about AI, but you also branch out into lots of other things like geopolitics and history and stuff like that. It's really great. And you're just one of the people that inspires me to make smart content. So I appreciate you coming on the show.

Dwarkesh Patel (00:02:25)

Yeah, that's very kind of you to say. I mean, I've always been sort of trying to have the conversations that I would like to have— If I was getting dinner with one of these professors or CEOs, what would I want to ask them? And I'm glad other people enjoy them as well.

Dan Shipper (00:02:38)

Yeah, it comes through. And I think it's really fun to get to turn the tables on you a little bit. You've done some interviews, but mostly you're interviewing other people. And I think it's probably on people's minds how you use AI in your work and in your life. And so that's what we're going to talk about today. So maybe just start by giving us a little bit of an overview. How is AI integrated into your work and in your life right now?

Dwarkesh Patel (00:03:06)

Yeah. So, it's actually changed a lot. I remember a year ago—I think it was after GPT-4. Somebody asked me, do you use AI to help you with your research or prep? And I was like, not at all. It's completely useless and mid. It gives you these banal— You ask it, what should I ask? So, it's a professor—and it'll give you these banal— Where did you grow up? What's your book about? Whatever. So initially it was terrible. I think recently, the models have gotten to just the point where, with [GPT-] 4o, or especially with the new Claude models, they're intelligent and interrogative and can consider the context which you provide to them. And so they're still not that good at it. What should I ask this person? Because obviously that's why I have a job, right, so that I can come for the questions.

But for the research itself, for me, at least I try to ingest everything they've ever written—all the rebuttals to their ideas, all the other considerations, and there's often a lot involved. Especially given there's many different fields I try to go deep into. The last interview I just did was with Dylan Patel who writes SemiAnalysis. It's a publication about semiconductors and AI hardware and so on. So there's a bunch you have to learn and I can go through my workflow, but it's incredibly useful to be able to have this thing where I'm like, what's going on here? Can you help me explain this? And then I guess one bigger thing I've been thinking about is ever since I interviewed Andy Matuschak. If your audience is familiar, he's the guy who talks a lot about how spaced repetition and other tools can enhance our ability to learn and how the normal mode of learning, you're actually not picking up that much. If you pick up a random book and start reading, you're not getting that much out of it.

And I really have found that to be the case, to the extent that, if I'm just casually reading a book, I think I'm basically wasting time or entertaining myself, and I have come up with a couple of different workflows and tools that help me really interrogate and make sure I've reinforced what I'm reading about or learning. And a tool like a language model is very helpful because it gives you the content in another context. It can quiz you if you want. So it's super helpful with that kind of stuff.

Dan Shipper (00:05:28)

That's really cool. I want to start back to front with the stuff you're using it to read. Because I think all that reading is one of the inputs to the interviews, and then we'll get into the interviews. And I'm really excited for both. So let's start with using AI to read and to learn.

Dwarkesh Patel (00:05:43)

Yeah, I can show you my screen, if that's right? So as I was talking about, one of the main things I think is important is, if I'm studying a topic over the course of a few weeks, especially if it's a difficult topic, it's new to me, it's incredibly important that I'm not just casually reading. Because if you're just casually reading, every day you're rereading the same key terms, the same concepts in your start over from scratch.

So one of the things I like to do, for example, I was recently interviewing Dylan, right? So if I go to his publication, SemiAnalysis, there’s just a ton of lingo and things you have to understand. So, the new one was pretty interesting. It's talking about why nobody has built a huge training cluster yet. And then the first thing I do is just, what are the key ideas and concepts I really need to understand? So I made myself a HuggingFace space. You honestly don't need to do anything like this. It's pretty simple to have Claude build you a HuggingFace space, or if you prefer, what it literally does is like apply this prompt to everything I paste in. So you can just copy-paste that prompt into Claude yourself, but basically, I copy-pasted some of the things in Andy Matuschak's post about how to write good prompts and I just asked Claude to make those prompts for me—spaced repetition prompts.

So when I do this, hopefully in a few seconds, we'll get something back. Initially, this will just give me some ideas of, what are the key ideas here I need to understand? So, super useful, right? I can zoom in a little bit. So, it's more helpful. So, for the audience who's listening, it's given me a bunch of question-answer pairs that consolidate the key things I need to understand about this post. We can go through the specifics here. I'm sure that the actual specifics of AI hardware will bore people, but a lot of the things where it's like, okay, if you don't get this, you've totally missed the boat here. And so you can start with something like this.

I added it to my spaced repetition app. Or, I can just look through this and I'm getting a sense of like, oh, okay. Here's what it would take to train a GPT-4-level model on a 100,000 H100 cluster. What are the three main types of parallelism you need to use to train on a big cluster or whatever? And this is for a technical post on other kinds of posts. There might be different kinds of cards that come up in history. It might be a different kind of thing for philosophy. It might be a different kind of thing. So this gives me a lay of the land. I'm sorry—you were going to say something?

Dan Shipper (00:08:15)

No, I love this. This is super interesting. I feel like I can go in a bunch of different directions, but where I want to start is how are you reading and when are you reading? So, are you using this specifically for reading that you're doing for the show or are you just doing this for any reading that you're doing that you feel like is serious and you really want to learn?

Dwarkesh Patel (00:08:33)

Both. So just this weekend I was reading. I forgot the author's name, but it's a book called Medieval Technology and Social Change. And it's about how different things that were developed through the last 1,500 years—technology like the stirrups, how they affected society. And it's entertaining. You can read it. And then one of the things is, okay, did I really understand what's going on here with the relationship he's trying to elucidate? So afterwards, in fact, I have some Claude chats where I was just going through while I was reading it. Let’s see if this recollects, I'm curious. I don't know how interesting it will be to your audience for me to go through point by point.

Dan Shipper (00:09:21)

Do it. I want to know. I'm on the edge of my seat because I have this book. It's sitting on the desk in front of me. And so I want to know what you got out of it.

Dwarkesh Patel (00:09:21)

Okay. So, first, I would just ask it to make some spaced repetition prompts for me. First of all, I read the chapter. I'm not sure I got it. So, just explain to me the chapter about how he says that stirrups created feudalism. What exactly was the connection here? So it's much more condensed here's what's going on here. Basically, if you understand this, it's a useful scaffold so that when you're reading the rest of the chapter, you understand where the pieces fit together. Then I added some—

Dan Shipper (00:09:58)

Have you tried— One of the things that I've tried with this is, because sometimes it doesn't know, especially for a book like that, where it's not that popular. Have you tried— One of the things I do is create a little Claude project and then upload the text if I can find it. Have you tried that?

Dwarkesh Patel (00:10:14)

In fact, let me just— Claude.ai, projects. So, I literally just think— I'm a host of a podcast where I try to ask good questions. My upcoming guest is a geneticist, and I just upload— I get the EPUB of the file. I convert the EPUB to a text using an online converter. I uploaded it to Project Knowledge. Then I've only just started prepping for this guest, but I'll just have a bunch of chats where I'm like, how does he explain what groups made up modern Europeans? It has all the context in there. That ends up being incredibly useful like you were saying.

Dan Shipper (00:10:51)

Yeah, that's so cool. I love that. I love that feature. Okay, wait, let's go back to stirrups and this chat you're having with this book.

Dwarkesh Patel (00:11:12)

Yeah. So it explains that the reason stirrups should create feudalism is because you needed a lot of land, basically to support the kinds of people who become heavy cavalry—the knights need a lot of land in order to have the income to have armor and lances and other kinds of equipment and to train themselves. But a knight is only possible if you have a stirrup against which you can brace yourself as you're attacking with a sword. Because otherwise you're just a Mongol who's shooting bows and arrows. But then there's a bunch of stuff that's confusing here. Why is it so expensive to have to be a knight that you need to completely confiscate church lands in order to subsidize this knight lifestyle? And then on these kinds of questions, the author is dead. But I'm just murky about it. I don't know what's going on. So, I can do just these kinds of things the book didn't even talk about, right? But I can always just continue the conversation with Claude and have it explain what's going on. And so this is just a recreational reading that Claude ends up being super helpful with.

Dan Shipper (00:12:13)

I think that's really interesting. What do you think about books like this as a person who likes history a lot—books that sort of single out a specific thing like the stirrup, and then are like, well, you can trace all this stuff to that one thing where it makes so much sense. But then there are things like, I don't know, Guns, Germs and Steel where like Jared Diamond had that whole thesis about—I can't remember the exact thing, but it's like people in warmer climates, or I can't remember the exact things. But it turned out to be totally wrong. How do you feel about things like that?

Dwarkesh Patel (00:12:45)

Yeah. So my opinion on these kinds of books: There's the sort of concise answer is, yeah, there's ones that do it poorly, but just don't read the ones that do it poorly or something. There is a failure mode for public intellectuals where they initially start off with a discipline and they do some exemplary work there. And then they write an initial broad book that's about how this idea explains a lot of the world and it does incredibly well. And now they're in public intellectual mode. And now the next book has to be, here's my theory of everything. And it's just not that satisfying. So I do worry about those kinds of things, but presumably the reason—I don't know, I'm not into reading 500-page books about things like how this chair physically worked. What's the point of that, right? I do want to understand the implications and maybe they're wrong, but what else are we trying to do here, right? Maybe you just intrinsically care about how the stirrup physically works there? I will point out a couple of examples.

So there's a lot of interesting topics where you really can't get at the heart of the matter without just considering the whole story. And in fact, a couple of biographies especially stand out in this way, where if you look at Caro's biography of LBJ or Kotkin’s biography of Stalin, it's basically a history of the 20th century or, in the case of Kotkin, even before the 20th century, I think the Caro books on LBJ start off with the Comanche raids on frontier settlers in the mid-19th century or something. And it goes through rural life in Texas—why electrification was such a big deal, a whole bunch of other things, right? Now it's basically a history of the 20th century, but it has a very specific point of view or a specific locus, a character that's moving the story along. And I find those to be incredibly helpful in getting a full picture of what's going on in an era.

There's a couple other books where they really aren't trying to write a theory of everything. I don't think Caro’s trying to write about the history of the 20th century, but they just can't help themselves. They feel like you really cannot understand the very specific topic I care about unless I tell you everything about everything like, Kotkin's story biography of Stalin starts with Bismarck's career as a military general and how that changed the way that different powers thought about colonialism and the need to modernize. And that's where it starts, right? And it's a biography of Stalin. So yeah, I love those kinds of books.

Dan Shipper (00:15:17)

I think there's a very deep point about the universe being interconnected there, but there's also a really interesting point for people who want to make stuff—write or make podcasts or whatever—because there's this deep fear that everyone has about being pigeonholed. And it's like, well, if I pick this really specific topic, I won't be able to bring all of myself to it. I won't be able to be multifaceted. And it's like, no, no, no. If you just pick one guy, Lyndon Johnson, and really get deep into him, you have to explain everything else about the world in order to explain him. And I love that. And, as a creator myself, that's the thing that I think about when I'm like, oh, maybe I'm getting too narrow here. It's like, no, no, the narrow is actually good. You can find the entire universe in the narrow.

Dwarkesh Patel (00:16:00)

Yes. I couldn't have said it better.

Dan Shipper (00:16:02)

So basically what I'm seeing right now is you're using Claude when you're reading books that you care about learning from, and you're using it a little bit to like prepare your mind for what you're about to read, which I think is a particularly good for like difficult books or for thinking through a particular argument before you go through it, you're asking questions. So it's a reading companion, you're getting more out of the books you read from that. But then you kind of take what you've read and throw it into this card generator.

Dwarkesh Patel (00:16:36)

Yeah. And so that mostly it's just chatting with Claude. And so, let me see if I can find a better example. So, I mean, a lot of topics I just find I've had a vague sense of what's happening, but I don't really get it. And it's super helpful to chat with Claude to make sure I'm on the right track. Dylan has a couple of posts about why packing is a technology super necessary for these advanced chips. I'm not trying to make this podcast all about AI hardware. It just happens to be the last podcast I did. So that's what you're getting. But it's confusing, it's five series posts about how advanced packing works and what the technical specifications are. And I'm like, wait, step back. Why is this necessary? What's going on? All kinds of other questions about when there's questions about how I'm worried about where I might get too deep in the weeds when I'm just explaining. Yeah, basically, I'm just like, how do I think about the broader context of what's happening here? Because I really can't ask good questions unless I have a good mental model of what's going on, what they're talking about. I really get where all this fits together.

Dan Shipper (00:17:56)

That makes sense. And so, Claude is kind of the first thing you flip to when you want to know that you are using it on mobile or using it on desktop?

Dwarkesh Patel (00:18:05)

Desktop.

Dan Shipper (00:18:06)

Okay. Interesting. So you're doing most of your reading and research stuff on the desktop.

Dwarkesh Patel (00:18:10)

Yeah, that's right.

Dan Shipper (00:18:11)

Hmm. And what do you think about Claude being really great right now and— I assume your ChatGPT usage is lower than it used to be?

Dwarkesh Patel (00:18:20)

Yeah. I think these things will keep getting better over time and I think we're just getting in the practice of using these tools. I'll talk a little bit about how these tools relate to my personal life. The post-production process initially was kind of useless, but I did spend a few weekends trying to write a few prompts and create a workflow at the time. It was basically useless. Now it's actually ended up being useful and I can use the same Jupyter notebooks or whatever to get things done. So it is worth investing, even if they don't work perfectly now, to get them part of your workflow so that as they keep getting better, you're getting the returns from that.

Dan Shipper (00:19:00)

Yeah, that makes sense. So, I want to just go back to the Anki card generator, the spaced repetition card generator. So as part of this, once you've done all of that clearing the ground conceptually for yourself to kind of understand the basics of what a guest is talking about or an idea that you're interested in, then you're kind of you're adding to your flashcards, I guess so that you retain the information past even when you talk to that guest. Is that right?

Dwarkesh Patel (00:19:30)

Yes, that's right. I mean, I think the larger mission of the podcast is to consult. Why does the podcast get better over time? And it's because basically I'm getting smarter or learning more things. I'm reducing my ignorance around a bunch of topics. And so if I don't do that I think about all the episodes I did before I interviewed Andy and started using spaced repetition and I just really regret it because I talked to all of these world experts in a ton of different domains. And to be honest, in many cases, I didn't take that much away. I vaguely remember some things. And now that I use it, I can walk you through the kinds of cards I make in these very separate tools I use, but it's totally a game changer in terms of what I can retain. In fact, I think it's not even about making sure I remember what I discussed in a previous episode or what I learned previously. It's more about future learning because I'm sure you've heard the saying about a learning compound because you can use what you've learned in the past to learn future things because they all interconnect. Well, you can't do that if you basically forgot most things you've learned in the past. So, yeah, my learning has a future of other things has become much faster because I have cashed all these different concepts and figures and facts. And so I understand how everything fits together much more. It's not even about the past. It's really about future learning.

Dan Shipper (00:21:00)

I don't know what you use for spaced repetitions. Can we see your deck?

Dwarkesh Patel (00:21:02)

I will point out by the way, as a side note, one use case of Claude that ended up actually being pretty useful. Sometimes you read obscure for a lot of light— I was reading Nick Land’s selected writings about AI and his acceleration and I was like, what's going on? Genuinely, what is his argument? Basically, why does he think that the AI takeover and whatever thing it creates in the aftermath will be good? Because he's a smart guy. I'm assuming he has an interesting argument. So I pulled the PDF of his selected writings. I just asked Claude okay, so why does he think it's a good thing that AI takes over humans? It offers a summary—initially this isn't necessarily that helpful because I kind of did read this in the essay, but what's helpful is that when you go through and I'm like, I respond, I don't get it. What does he think is wrong with human society that you have to erase it? And then he gives an explanation. I'm like, I still don't get it. What exactly are you talking about here? And then here's what I do with the podcast, right? I have the guest on and I ask them, what do you mean here? I disagree. Here's a contradiction—whatever. And going through their writings with Claude and, have I actually found a sort of blind spot in their thinking? Or is this just me being confused by their ideas? It's super helpful.

Dan Shipper (00:22:19)

That is really interesting. It's like you can get down to a deeper level before you talk to them so that you can start there with them as opposed to starting at the surface, which is really cool. I use that too for difficult books, not necessarily for interviewing the author of those books, but, for example, I interviewed Reid Hoffman, I don't know, a month or two ago. And I wanted to talk to him about the kind of intersection between philosophy and AI.

And he almost became a philosophy professor at Oxford, and was really deep into Wittgenstein. So I read a bunch of Wittgenstein which I hadn't read in a while, and I just used Claude for it, and it was so much better because I haven't taken a Wittgenstein class—or maybe I took one in college a long time ago, but I've read him a lot and there are always those points in those kinds of books where you're like, I think I know what they're saying, but, I'd probably have to go to a graduate school and get a master's in this to really know. And Claude actually makes me be like, oh, I don't need that anymore. Any book I want to read, I basically know. And it just helped me so much in that interview because I could just ask, read really deep Wittgenstein-related questions and he could answer them.

Dwarkesh Patel (00:23:36)

Yep. I think that's totally legitimate. I think some people would be like, oh, you need to read it in the original blah, blah, blah. I think if you care about the ideas and you think the ideas are timeless and not the ideas are not about the specific kind of pros that the original author used, but just generally what is the essence and the gist of what's happening here. If you care about the ideas and I think this is totally valid, right? I don't just agree with the people who are like, no, you need to read the specific syllables that Wittgenstein used.

Dan Shipper (00:24:05)

Yeah. I mean, I'm also just saying, I have the book open and then I just take one of his statements and just throw it in there and then it's like, here's what it means or whatever, which I think is really great. Okay, so you're going to show us the spaced repetition card. So what app is this?

Dwarkesh Patel (00:24:20)

This is Mochi. It's like Anki, but this is the one I use.

Dan Shipper (00:24:25)

Why?

Dwarkesh Patel (00:24:26)

Actually, I don't have any cards today cause I just went through them this morning, but let me give you a sense of what kinds of things I have. So I have if you go through history, recently, I don't know if you can see my screen, how easy this is. Maybe I'll zoom in a little more. I've been planning on interviewing David Reich, who is a geneticist who explores human origins. And these are especially cases where just reading the book, I'm like, I would have totally forgotten. He names all these different ancestral groups and how they combine and in what eras when did the Yamnaya people come through Europe? When did the Anatolian hunter gatherers wash over Eurasia—all these things that were just like you read it in one ear, it goes out the other one, unless you make cards for it. And so I made a ton of cards about this kind of stuff. So there's examples of that here. It's especially useful for hardware and technical things. So here I feel like if I don't make cards, I'm just constantly relearning the same things cause I didn't learn the lingo in the right way.