GPT-4: A Copilot For The Mind (2023)

Large language models are perhaps the ultimate study buddy, that one thing that might help you actually absorb information while you’re reading. That’s Dan Shipper’s vision, as outlined in this essay from March 2023. Dan’s essay from last year stands out as a prescient reflection on what’s come since. And with OpenAI’s DevDay set for October 1 and Every taking a quarterly Think Week, we thought it was ripe to republish as part of our week of essays on the power of ChatGPT.

ICYMI: We created eight custom wallpapers based on Every’s art for iPhone or Android. Download them for free.—Kate Lee

Over the next year or two, I expect GPT-4 and its successors to become a copilot for the mind: a digital research assistant that will bring to bear the sum total of everything you’ve read, everything you’ve thought, and everything you’ve forgotten every time you touch a keyboard.

It will solve some of the perennial problems in productivity culture: remembering what you read, then helping you apply it to your writing, your business, and your life.

It will bring back the ideas, quotes, and memories you need, when you need them most, with no organizing, tagging, or linking required. It will work as a personalized extension of your intelligence available 24/7 at the touch of a button.

I’ve written about this a few times in “The End of Organizing” and “Where Copilots Work,” but this week it’s clear that the dominos are starting to falling into place: 

• GPT-4 sports an 8x larger context window (the main thing bounding copilot use cases is small context window sizes).
• Microsoft is building copilots into all of its 365 products. It aggregates all of your notes, documents, and meetings together to help you autocomplete memos, emails, and spreadsheets.

It’s still early, and these technologies will need a lot of work before they are ready for prime time. Impressive demos don’t equal actually useful software.

But in this essay, I want to explore in more detail the problems that I think this copilot for the mind might solve, and what’s feasible today with the advent of GPT-4.

Let’s dive in.

The problem with reading

I love reading, but I don’t remember much of what I read. I’ve tried all sorts of hacky solutions: highlights, Zettelkastens, Anki cards, book notes, and more.

The fundamental problem is that I don't know when I’m going to need a particular fact, quote, or idea again. So all of these strategies are aimed at either improving my memory to keep them top of mind, or creating an organizational strategy that makes sure I bump into them later when I need them.

To some degree, the solutions I’ve tried so far have worked—but they only take you so far. I go on and off with taking reading notes and building a Zettelkasten. It’s helpful, but it’s also a lot of work, and I don’t find myself referencing it that often.

Anki cards provide some glimmer of hope here because they purport to load your brain with the knowledge you want, rather than keeping it in an organizational system. But I find that Anki makes me good at remembering the answers to Anki cards—rather than bringing the knowledge contained in them into the world and into my writing.

What I really want is to be steeped in these ideas and in the language of these writers, thinkers, and artists all day—instead of just at night when I’m reading before bed. (There’s a reason why some of the best ideas come out of the university environment. It’s far easier to reference the eminent dead when that’s what everyone around you is also doing.)

The goal here is erudition and application. I want to be able to write referencing the best ideas that have come before me. And I want to be able to make decisions, and see the world through the lenses provided by the people I’ve read. In short, I want to participate as fully as possible in the intellectual and emotional evolution of humanity.

AI as copilot for the mind

When you have extra intelligence at your disposal, you don’t need to worry so much about remembering what you read. Sure, it’s better if you have a good memory. But an AI-powered copilot for the mind is going to beat a memory palace technique or even a genius-level IQ every day of the week.

Here’s how it might work:

Every time you touch the keyboard, the AI downloads and sorts through everything you’ve ever read and uses the sum total of that knowledge to help you complete your sentences.

It’ll work in every app that you use, and it will be on your computer and on your phone. And it’ll be incredibly easy to save anything that you want to remember for later. Just take a screenshot, or paste a URL, or upload a file, or send a text, and whatever you’re looking at will be saved and available for your copilot to bring back to you at the right time.

A copilot for the mind might be able to find the perfect quote from a book you highlighted 10 years ago to help you make the point you’re trying to make. Or it might suggest relevant counterarguments to help you sharpen your prose—or maybe even change your mind on a topic you’re thinking about.

If you’re anything like me, you’ve spent some portion of the past 10 years fascinated with mental models—trying to apply the best of science and behavioral econ and literature to your day-to-day to try to make better decisions. The problem is, creating a list of mental models doesn’t really help you apply them particularly well.

A copilot for the mind can help with that too. If you’re struggling to make a decision, it could suggest inverting the problem, or help you see when you’re struggling with the sunk-cost fallacy. It makes mental models practical and actionable—instead of just aspirational.

The key thing to note here, though, is that the ideal copilot isn’t just referencing any relevant book or fact when it tries to help you. It’s referencing your books and your notes when you’re working with it. Here’s why.

Your intellectual history will power your copilot

In Jorge Luis Borges’s short story “The Library of Babel,” he creates an infinite library that contains all possible books. Contained somewhere in this library are just the sort of books every book nerd wants to read: a book that predicts your future accurately, a book that unifies quantum mechanics with general relativity, a book that reveals the sociology of alien life on other planets, a book that pins down exactly what love means and what is at the heart of human nature, a book that contains the one true moral system by which to live.

But again, this library contains every possible book. So it also contains a lot of gibberish. Most of the books, in fact, are complete gibberish. This makes navigating the library an impossible task—and so even though it contains the secrets of the universe, it’s unusable by mortals. 

On their own, large language models are, to a significant extent, Babel-like. Their latent space can output every possible combination of words. They are capable of creating genius-level sentences—and also false gibberish. And at this point in the life cycle of this technology, the quality of the results you’re going to get is far higher when they’re grounded in a knowledge-base for the AI to reference when it is trying to respond to your prompts.

That’s why if you’ve spent the hours carefully curating your own personal library—whether that’s books, or articles, or videos, or movies—all of that time will significantly improve your copilot experience.

For one, it creates a base of trusted information for the copilot to use. It won’t be able to reference anything that you haven’t looked at—so it will be far less likely to return something that’s untrue, or out of date.

For another, because you’re usually consuming information that’s related to whatever you’re thinking about, giving a copilot access to your personal knowledge base is going to help it return results that are actually relevant to you. 

While a copilot could technically cite an author that you’ve never heard of while you’re typing out an essay or making a decision, that’s probably not going to be too helpful. Instead, it would be far better for it to use ideas and thinkers that you’ve already engaged with on your own, so that its completions can click easily into your brain without too much thinking.

If you find this vision of the future compelling, the next question is: How feasible is it actually?

How feasible is this now, and what are its limitations?

If we parse through the hype, there are a variety of technical limitations to this technology. Today’s AI tools produce “wow” moment demos, but they’re going to require continual refinement over a period of years before they’re reliable and practical enough to be software we use every day.

Here are a few of the current limitations that need to be overcome for this kind of experience to be built:

• Context window limitations. Models like GPT-4 can only read through a set amount of information at a given time. The bigger the context window, the more information it can use to complete a query. GPT-4’s context window is 8x larger than GPT-3’s—so this is improving dramatically. 
• Privacy and IP concerns. Many users are going to be hesitant about uploading notes or highlights or journal entries to models like these—for good reason. I suspect these use cases will start to take off when high-quality LLM experiences are available to run natively on your phone or laptop, instead of forcing you to send your data to a cloud API for completion.
• An actually good user experience. What you want is a UX where copilot completions are shown in a frictionless way that feels helpful instead of annoying. GitHub CoPilot nailed this for programming, so I believe it’s possible for other use cases. But it’s a balancing act. For more, read my essay "Where Copilots Work."

The problems I listed above are very real—and they could hold back progress on these types of tools for a while. But I think the rate of improvement in this technology is high enough, and demand is real enough that eventually we’ll see tools like this proliferate.

A peek into the future

My house is filled with paper books; it’s a floor-to-ceiling affair. I have books in the bedroom, books in the living room, and books piled on top of the cabinets in my kitchen. There’s just something about a physical paper book that I am in love with.

These stacks of books track my intellectual journey through life. There’s the Russian novels I loved in high school, the philosophy of science phase from college, the books on complexity that I devoured when I was in my early 20s, the psychology and spirituality books I discovered in my late twenties.

My Readwise, too, is filled with highlights. My Roam is bursting with ideas, quotes, and journal entries. So is my Evernote, my Notion, and my paper notebooks.

All of these things are containers for bits of my life. Right now those containers are lifeless, and everything that they contain is locked away.

AI is going to unlock these containers, and bring them to life. And it’s coming sooner than you think.