"a boy seen from behind, looking up in wonder at a shelves and shelves of books in a library in front of him, romantic, golden lighting"

GPT-4: A Copilot for the Mind

Never forget what you read again

261 5

Sponsored By: Reflect

This article is brought to you by Reflect, a frictionless note-taking app with a built-in AI assistant. Use it to generate summaries, list key takeaways or action items, or ask it anything you want. 

Over the next year or two, I expect GPT-4 and its successors to become a copilot for the mind: a digital research assistant that will bring to bear the sum total of everything you’ve read, everything you’ve thought, and everything you’ve forgotten every time you touch a keyboard. 

It will solve some of the perennial problems in productivity culture: remembering what you read, then helping you apply it to your writing, your business, and your life. 

It will bring back the ideas, quotes, and memories you need, when you need them most, with no organizing, tagging, or linking required. It will work as a personalized extension of your intelligence available 24/7 at the touch of a button.

I’ve written about this a few times in “The End of Organizing” and “Where Copilots Work”, but this week it’s clear that the dominos are starting to falling into place: 

  • GPT-4 sports an 8x larger context window (the main thing bounding copilot use cases is small context window sizes)
  • Microsoft is building copilots into all of their Office 365 products. It aggregates all of your notes, documents, and meetings together to help you autocomplete memos, emails, and spreadsheets

It’s still early, and these technologies will need a lot of work before they are ready for prime time. Impressive demos don’t equal actually useful software.

But in this essay, I want to explore in more detail the problems that I think this copilot for the mind might solve, and what’s feasible today with the advent of GPT-4.

Let’s dive in.

The problem with reading

I love reading, but I don’t remember much of what I read. I’ve tried all sorts of hacky solutions: highlights, Zettelkastens, Anki cards, book notes, and more.

The fundamental problem is that I don't know when I’m going to need a particular fact, quote, or idea again. So all of these strategies are aimed at either improving my memory to keep them top of mind, or creating an organizational strategy that makes sure I bump into them later when I need them. 

To some degree, the solutions I’ve tried so far have worked—but they only take you so far. I go on and off with taking reading notes and building a Zettelkasten. It’s helpful, but it’s also a lot of work, and I don’t find myself referencing it that often. 

Anki cards provide some glimmer of hope here because they purport to load your brain with the knowledge you want, rather than keeping it in an organizational system. But I find that Anki makes me good at remembering the answers to Anki cards—rather than bringing the knowledge contained in them into the world and into my writing.

What I really want is to be steeped in these ideas and in the language of these writers, thinkers, and artists all day—instead of just at night when I’m reading before bed. (There’s a reason why some of the best ideas come out of the university environment. It’s far easier to reference the eminent dead when that’s what everyone around you is also doing.)

The goal here is erudition and application. I want to be able to write referencing the best ideas that have come before me. And I want to be able to make decisions, and see the world through the lenses provided by the people I’ve read. In short, I want to participate as fully as possible in the intellectual and emotional evolution of humanity.

Imagine combining ChatGPT with Apple Notes. That's what it feels like using Reflect – an ultra-fast notes app with an AI assistant built in directly. Use the AI assistant to organize your notes and thoughts, improve your writing and boost your productivity.

Reflect also uses Whisper from OpenAI to transcribe voice notes with near human-level accuracy. That means you can use Reflect to ramble about a topic, and then have the AI assistant turn it into an article outline. 

Start a free trial with Reflect to transform your note-taking using AI. 

AI as copilot for the mind

When you have extra intelligence at your disposal, you don’t need to worry so much about remembering what you read. Sure, it’s better if you have a good memory. But an AI-powered copilot for the mind is going to beat a memory palace technique or even a genius-level IQ every day of the week.

Here’s how it might work:

Every time you touch the keyboard, the AI downloads and sorts through everything you’ve ever read and uses the sum total of that knowledge to help you complete your sentences. 

It’ll work in every app that you use, and it will be on your computer and on your phone. And it’ll be incredibly easy to save anything that you want to remember for later. Just take a screenshot, or paste a URL, or upload a file, or send a text, and whatever you’re looking at will be saved and available for your copilot to bring back to you at the right time.

A copilot for the mind might be able to find the perfect quote from a book you highlighted 10 years ago to help you make the point you’re trying to make. Or it might suggest relevant counterarguments to help you sharpen your prose—or maybe even change your mind on a topic you’re thinking about. 

If you’re anything like me, you’ve spent some portion of the past 10 years fascinated with mental models—trying to apply the best of science and behavioral econ and literature to your day-to-day to try to make better decisions. The problem is, creating a list of mental models doesn’t really help you apply them particularly well.

A copilot for the mind can help with that too. If you’re struggling to make a decision, it could suggest inverting the problem, or help you see when you’re struggling with the sunk cost fallacy. It makes mental models practical and actionable—instead of just aspirational.

The key thing to note here, though, is that the ideal copilot isn’t just referencing any relevant book or fact when it tries to help you. It’s referencing your books and your notes when you’re working with it. Here’s why.

Your intellectual history will power your copilot

In Jorge Luis Borges’ short story “The Library of Babel,” he creates an infinite library that contains all possible books. Contained somewhere in this library are just the sort of books every book nerd wants to read: a book that predicts your future accurately, a book that unifies quantum mechanics with general relativity, a book that reveals the sociology of alien life on other planets, a book that pins down exactly what love means and what is at the heart of human nature, a book that contains the one true moral system by which to live. 

But again, this library contains every possible book. So it also contains a lot of gibberish. Most of the books, in fact, are complete gibberish. This makes navigating the library an impossible task—and so even though it contains the secrets of the universe, it’s unusable by mortals. 

On their own, large language models (LLMs) are, to a significant extent, Babel-like. Their latent space can output every possible combination of words. They are capable of creating genius-level sentences—and also false gibberish. And at this point in the lifecycle of this technology, the quality of the results you’re going to get is far higher when they’re grounded in a knowledge-base for the AI to reference when it is trying to respond to your prompts.

That’s why if you’ve spent the hours to carefully curate your own personal library—whether that’s books, or articles, or videos, or movies—all of that time will significantly improve your copilot experience. 

For one, it creates a base of trusted information for the copilot to use. It won’t be able to reference anything that you haven’t looked at—so it will be far less likely to return something that’s untrue, or out of date.

For another, because you’re usually consuming information that’s related to whatever you’re thinking about, giving a copilot access to your personal knowledge base is going to help it return results that are actually relevant to you. 

While a copilot could technically cite an author that you’ve never heard of while you’re typing out an essay or making a decision, that’s probably not going to be too helpful. Instead, it would be far better for it to use ideas and thinkers that you’ve already engaged with on your own, so that its completions can click easily into your brain without too much thinking.

If you find this vision of the future compelling, the next question is: How feasible is it actually?

How feasible is this now, and what are its limitations?

If we parse through the hype, there are a variety of technical limitations to this technology. Today’s AI tools produce “wow” moment demos, but they’re going to require continual refinement over a period of years before they’re reliable and practical enough to be software we use every day.

Here are a few of the current limitations that need to be overcome for this kind of experience to be built:

  • Context window limitations. Models like GPT-4 can only read through a set amount of information at a given time. The bigger the context window, the more information it can use to complete a query. GPT-4’s context window is 8x larger than GPT-3’s—so this is improving dramatically. 
  • Privacy and IP concerns. Many users are going to be hesitant about uploading notes or highlights or journal entries to models like these—for good reason. I suspect these use cases will start to take off when high-quality LLM experiences are available to run natively on your phone or laptop, instead of forcing you to send your data to a cloud API for completion.
  • An actually good user experience. What you want is a UX where copilot completions are shown in a frictionless way that feels helpful instead of annoying. GitHub CoPilot nailed this for programming, so I believe it’s possible for other use cases. But it’s a balancing act. For more, read last week’s essay “Where Copilots Work.”

The problems I listed above are very real—and they could hold back progress on these types of tools for a while. But I think the rate of improvement in this technology is high enough, and demand is real enough that eventually we’ll see tools like this proliferate.

A peek into the future

My house is filled with paper books; it’s a floor-to-ceiling affair. I have books in the bedroom, books in the living room, and books piled on top of the cabinets in my kitchen. There’s just something about a physical paper book that I am in love with.

These stacks of books track my intellectual journey through life. There’s the Russian novels I loved in high school, the philosophy of science phase from college, the books on complexity that I devoured when I was in my early 20s, the psychology and spirituality books I discovered in my late 20s.

My Readwise, too, is filled with highlights. My Roam is bursting with ideas, quotes, and journal entries. So is my Evernote, my Notion, and my paper notebooks.

All of these things are containers for bits of my life. Right now those containers are lifeless, and everything that they contain is locked away.

AI is going to unlock these containers, and bring them to life. And it’s coming sooner than you think.


Find Out What
Comes Next in Tech.

Start your free trial.

New ideas to help you build the future—in your inbox, every day. Trusted by over 75,000 readers.

Subscribe

Already have an account? Sign in

What's included?

  • Unlimited access to our daily essays by Dan Shipper, Evan Armstrong, and a roster of the best tech writers on the internet
  • Full access to an archive of hundreds of in-depth articles
  • Unlimited software access to Spiral, Sparkle, and Lex

  • Priority access and subscriber-only discounts to courses, events, and more
  • Ad-free experience
  • Access to our Discord community

Thanks to our Sponsor: Reflect

Transform your note-taking with Reflect using our new AI assistant. Have it list to-do lists from your notes, rephrase your writing, and even generate counter arguments. 

Comments

You need to login before you can comment.
Don't have an account? Sign up!
@mattes over 1 year ago

Hi Dan—very good points. I think personal LLMs will have huge value for people and can change the industry again. You'll find my thoughts about this here: https://medium.com/@mattesmattes/chatgpt-4-may-be-the-best-thing-that-could-happen-to-apple-3432fb33ba12

Chris Harvey over 1 year ago

Dan, great article! I'm excited for the future.

One critique and one comment:

1) When you say the context window of [[ChatGPT-4]] is "8x larger" than Chatgpt-3.5, i think you need to quantify what you're saying. Graham Lipman (@glipsman on Twitter) nailed it:
—About 25,000 words, or 100+ pages of a novel (see https://twitter.com/glipsman/status/1635697739349790720?s=20)

2) I've been told it was useless to capture personal ideas, notes, articles, books and other relevant time-bounded items because we have no good way to harvest them. It's like pulling all this fish from the sea but having no way to find what you've caught to cook up, at least not at scale. Now we have a machine than can not only find what you're caught, but help you preserve and find the best fish to eat. Very excited that my efforts to preserve my ideas may end up not being wasted.

—Chris

Diego Escosteguy over 1 year ago

Great article. One minor error: it’s Jorge Luis Borges, and not Jose. A copilot might help - all of us! - with this kind of mistake.

@michaelelling63 over 1 year ago

What if they make you less smart because "you don’t need to worry so much about remembering what you read." One's intellect and ability to imagine is a function of how one associates and synthesizes retained knowledge. Jeff Hawkins does a good job mapping how the brain develops and works relationally. Judea Pearl writes about how we imagine and innovate. Sure you can ride an ebike, but you won't exercise your muscles and become a better cyclist and be more fit. There are innumerable examples where people have lost basic mental and motor skills as they rely on new technology. That said, a process that helps you expand or maintain the existing associations and mental mappings will be quite useful. I would certainly be interested in mining the last 9 years of extensive digital note taking and linking within my 44 evernote notebooks.

frederick carter over 1 year ago

@michaelelling63 thats a take to seriously consider.

Every

What Comes Next in Tech

Subscribe to get new ideas about the future of business, technology, and the self—every day