🎧 How Notion Cofounder Simon Last Builds AI for Millions of Users

TL;DR: Today we’re releasing a new episode of our podcast AI & I. I go in depth with Simon Last, the cofounder of Notion. We get into the practical engineering challenges of building a reliable AI product at scale, how he would rebuild Notion in the age of AI, and his thoughts on the evolving role of humans as AI grows more powerful. Watch on X or YouTube, or listen on Spotify or Apple Podcasts.

Notion cofounder Simon Last told me everything he’s learned from integrating an AI application into a platform that has over 100 million users.

Simon likes to keep a low profile, even though he’s the driving force behind Notion AI, one of the most widely scaled AI applications in the world.

In this episode, we get into how AI changes the way he builds software since the days he cofounded Notion with Ivan Zhao in 2013. He talks about the challenges that arise because AI doesn’t follow the deterministic rules of traditional software, and how he designs evals to build AI systems that are reliable at scale. Simon tells me about the AI tools he uses to code and how he would think about rebuilding Notion from scratch with them. He also shares his thoughts on how the growing capabilities of AI are redefining human roles, and argues that we have the responsibility to shape technology to align with our collective vision of the future. Here’s a link to the episode transcript.

This is a must-watch for anyone interested in building reliable AI products at scale.

Watch on X or YouTube, or listen on Spotify or Apple Podcasts.

If you want a quick summary, here’s a taste for paying subscribers:

How AI reshapes the way we build software

Notion is a tool that helps people design their own workspaces by giving them the building blocks to organize and connect information in flexible ways. In technical terms, these building blocks—the simplest unit a system is built on—are known as “primitives.” Here’s how Simon thinks AI is changing the way we think about them:

AI is bringing new primitives into existence. Simon believes that AI is creating new categories of primitives, namely, LLMs themselves, which are “thinking box[es]” that you can give context to and instruct to perform different tasks, and embeddings (a technique which converts words into a language computers can understand) that enable “really good semantic search.”
Enhance traditional primitives with AI. The primitives that existed before AI remain important; they’re simply more useful with AI. “You can plug in ‘thinking boxes’ on top [of the primitives] to automate some of the tasks that a human would do in the past, especially things that are cumbersome,” Simon says.
Why permission models are important. According to Simon, having a “permission model”—a framework that defines who (or what) has access to different parts of a system and what actions they can perform—is even more relevant in the age of AI. It’s crucial that the “user can understand and control what the AI can read or write.”
Chat as a gateway to better interfaces. Chat is the most popular way we interface with AI, and while Simon believes it will continue to be relevant in some form, he thinks it will evolve to present users with custom solutions. “Chat is a low-level primitive that it makes sense to have, but the real goal is to connect people to some workflow or use case that's solving their problem.”
Develop high-fidelity ways to communicate. When users interface with AI through chat, they can prompt it to make complex changes based on high-level instructions, creating the challenge of helping users understand what changed and why. Simon frames it as a question of “what’s the thing that's changing, and what’s the most efficient and understandable way to present that.”

Beyond the foundational elements of software, AI has introduced a new way to analyze data. Traditionally, relational databases—a way of storing data in a structured format—have been widely used to manage data, and they worked best when the data's purpose was clearly defined upfront. For example, building a relational database for a customer relationship manager (CRM) is straightforward because the primary goal is known: tracking customer information through defined tables and relationships. However, embeddings can capture many more dimensions of a piece of information, perhaps making it better-suited for situations where future use cases aren't yet defined. These are Simon’s thoughts on the implications for Notion:

Purpose-driven design for databases. Simon has been playing with AI to create relational databases, and finds it most useful when prompted to design a minimal framework that’s required for the task at hand. “Don't design schemas that you don't know what they're for yet…each property should have a purpose that just really focuses the task and makes it more effective.”
Match tools to the task at hand. Simon would choose embeddings over deterministic forms of querying for open-ended questions, while preferring SQL-like queries when a precise answer is necessary. “I think of them as two different tools that you have in your toolbox—ideally you have both, and you can even maybe combine them.”

Simon’s strategy for building a scalable AI application

While traditional software follows deterministic rules—the same input always produces the same output—AI is stochastic, meaning the same input can produce different but probable outputs, like rolling a weighted dice. In this context, Simon explains the challenges of shipping Notion’s AI features at scale: “There’s the problem of ‘it can fail in some cases,’ but then there’s an additional meta problem—you might not even know the cases where it can fail…I've definitely been led astray multiple times thinking that I'd solved it, and then finding this whole new class of errors that are really hard to solve.”

This can make designing good evaluations difficult. Whenever possible, Simon prefers using “deterministic evals,” tests where you always expect the same outcome if you use the same input, over “non-deterministic ones,” where the outcome may vary even if you use the same input because of inherent uncertainty or variability. Here are his tips for reliable evals:

For deterministic evals of “complex workflows,” Simon likes to use “classifiers,” or tools that sort things into different groups based on patterns it has learned, to break the workflow up into simpler parts.
For non-deterministic evals, Simon recommends being “very clear” and “narrow” in defining the task you want to design an eval for, and then using a model which is “extremely good at that task, to the point where you can actually trust it.”

Simon mentions that his “job description” is to create good interfaces for LLMs. These are the principles that guide his work:

Speak the model’s language. According to Simon, aligning prompt formats with a model’s training data leads to improved responses. For example, at Notion, they discovered that LLMs performed better when the prompts were in Markdown formatting—“the language the model knows”—instead of their native XML, because the models are trained on Markdown.
Simple prompts win. Simon found that using simpler formats to prompt AI models worked better than trying to replicate the complex XML structures they used internally—although simplifying meant some loss of detail, it made responses more reliable and easier to control. His principle around this is: “Make it as simple as possible while still doing the tasks that I care about.”
Design out common errors. If you notice that a broad category of issues crop up often with a certain prompt, Simon recommends trying to make those errors impossible through system design, and only then focusing on improving the prompt itself. “My first line of defense is I want to try to make that class of issues be impossible in the system or validation around it.”

How Simon would rebuild Notion in the age of AI

Simon and his cofounder Ivan Zhao famously went to Kyoto in 2015 and, fueled by grit and bowls of cheap noodles, rewrote the code for Notion. As a thought experiment, I asked him what he would do differently if he could go back to that time and rebuild Notion with AI.

Simon believes that AI can automate cumbersome knowledge work tasks, where systems like CRMs, for example, would autonomously update by extracting information from channels like Slack and email. “In the AI world…hopefully the user interacts more with the processed outputs of [a database] rather than the raw database itself,” he said. Speaking to his style of working, Simon says he still likes to be “unencumbered by the way things work” when he’s thinking of a new project, while charting out an “incremental roadmap” to reach his ambitious goals.

These are the tools Simon currently uses (and the ones he’d presumably use if he had to rebuild Notion with AI):

AI code editor Cursor’s “autocomplete” feature, which suggests code completions based on context, and “retrieval over code,” which helps locate relevant code snippets in a code base
Claude’s Sonnet model is his “day-to-day workhorse.”
Simon has tried using OpenAI’s o1 and o1-mini to code, but finds that it’s “not that much better than Sonnet at coding” while also being “slow” and “finicky” to prompt because it’s been trained on data that has “little context and a lot of reasoning.”

Simon’s thoughts on the future of AI agents, AGI, and the value of humans

What will make AI agents effective. “When you can give it some high-level tasks, give it these tools…and then just enter a hard loop of think a bunch, use some tools, see the outputs, think more, and then keep looping on…and long-horizon reinforcement [learning] over that.”
How the role of humans will evolve as AI grows more powerful. “The most important thing is making sure that the shape of the future is aligned with what we care about…and a lot of [that] is orchestrating the tasks that the AIs are doing…and then once you have it doing that, making sure it's doing the right thing, and [having] a way to observe it and check on it, feels like the key.”
What human creativity and intuition means in a world with AI. “I don't think I assign a special value to humans being creative or intuitive…what really matters is…what you want to be done, and your ability to communicate that, and observe it, and keep it on the road.”
How Simon defines AGI. “I like OpenAI’s definition [of AGI] and just, ‘Can it do all the economically useful tasks?’ I think that’s the crucial question around the intelligence explosion.”
When will AGI become a reality. “I would assign a pretty decent probability to AGI in the next 10 years, and not necessarily requiring huge paradigm shifts.”
How the economy will change with AGI. “There's going to be a bunch of new stuff that we do, and it's going to create 100x more economic value than all the economic value that we have now…that comes from the fact that when you discover an economically useful task, with these models, if you buy more compute, you can just scale it up more—which is much more elastic than with humans.”

You can check out the episode on X, Spotify, Apple Podcasts, or YouTube. Links and timestamps are below:

Watch on X
Watch on YouTube
Listen on Spotify (make sure to follow to help us rank!)
Listen on Apple Podcasts

Timestamps:

Introduction: 00:01:57
How AI changes the way we build the foundational elements of software: 00:02:28
Simon’s take on the impact of AI on data structures: 00:10:07
The way Simon would rebuild Notion with AI: 00:13:05
How to design good interfaces for LLMs: 00:23:39
An inside look at how Notion ships reliable AI systems at scale: 00:28:22
The tools Simon uses to code: 00:35:41
Simon’s thoughts on scaling inference compute as a new paradigm: 00:38:16
How the growing capabilities of AI will redefine human roles: 00:49:10
Simon’s AGI timeline: 00:50:28

What do you use AI for? Have you found any interesting or surprising use cases? We want to hear from you—and we might even interview you. Reply here to talk to me!

Miss an episode? Catch up on my recent conversations with star podcaster Dwarkesh Patel, LinkedIn cofounder Reid Hoffman, a16z Podcast host Steph Smith, economist Tyler Cowen, writer and entrepreneur David Perell, founder and newsletter operator Ben Tossell, and others, and learn how they use AI to think, create, and relate.

If you’re enjoying my work, here are a few things I recommend:

Subscribe to Every
Follow me on X
Subscribe to Every’s YouTube channel

Thanks to Rhea Purohit for editorial support.

Dan Shipper is the cofounder and CEO of Every, where he writes the Chain of Thought column and hosts the podcast AI & I. You can follow him on X at @danshipper and on LinkedIn, and Every on X at @every and on LinkedIn.

We also build AI tools for readers like you. Automate repeat writing with Spiral. Organize files automatically with Sparkle. Write something great with Lex.