Apple’s AI Evolution: Built-in AI, Supercharged Siri, and Ecosystem Lock-in

Was this newsletter forwarded to you? Sign up to get it in your inbox.

1. Built-in AI: Apple will use a privacy-first, on-device approach to AI to inject magical new capabilities into its existing software.
2. Supercharged Siri: The voice assistant Siri is becoming the meta-layer of software, coordinating workflows and reducing the time you spend in applications.
3. Ecosystem evolution: Each of Apple’s devices—iPhone, iPad, Mac—works in concert with the other, making the sum greater than the parts.

Each of these elements is exciting on its own and, if the technology continues to deliver, will be a genuine improvement on how we work and live. For centuries, new technology has automated an increasing amount of the physical labor we previously did. It is time for it to do the same for mental labor, too. Soon, you won’t need to navigate to a website or app to experience generative AI—it’ll be built into everything you do on your smartphone and computer.

Mathematical magic (built-in AI)

The most impressive application of the day was the iPad calculator app. The company had previously promised not to make one for the iPad until it could “do something distinctly great.” It was a 14-year wait, but boy, did Apple deliver. 

The calculator is familiar to the form on your iPhone, sprinkled with touches of AI. You can write out mathematical expressions with the Apple pencil, and as soon as you write an equals sign, the calculator will solve it for you—in your own handwriting. It can create graphs in real time and adjust the output as you update the variables in the equations making the graph. It is beautiful and intuitive, the demon horror spawn of a math teacher’s nightmares—and it is all powered by AI calculated on-device. (It apparently drew the loudest cheers of the day). 

Beyond that one app, Apple demonstrated numerous instances of machine-learning capabilities enhancing native existing applications and hardware. Photos can now do semantic search with contacts. So I could type, “Morgan hugging Maple in front of a tree,” and the Photos app will pull up a picture of my wife with my puppy playing in front of an Eastern white pine. iMessage can generate custom emoji using image-generation models. Apple Watch can use machine learning to analyze the effort you put into a workout. The phone app can record and transcribe conversations—a boon for writers everywhere. I can do this all day—and name startups that these features kill.

The main takeaway is that the primary capabilities of generative AI—such as finding data in large data sets, semantic understanding, and image generation—are coming to all of your iPhone apps.

*Arnold Schwarzenegger voice* I’ll be automated (supercharged Siri) 

I’ve been arguing for a while now that the ultimate form of AI in software is as a "meta-layer" that automates your repetitive tasks that span multiple applications. As such, I am grateful to the good folks at Apple for proving me right. Siri is exactly what I said it would be.The AI update to Siri means that you can do all the things you want it to do. One example Apple shared in the presentation was the query, “What time does Mom land?” Siri can search your messages from Mom for when she shared her flight number, look up its progress, and tell you the updated arrival time. Siri can also manage notifications, emails, and messages, summarizing long messages and surfacing the most important information.

Source: Apple.

As you respond to that incoming information, Siri can generate text or take actions for you in the applications. So, Siri can compose your answers in emails or respond to texts for you. These are capabilities we’ve seen in AI apps before, but never performed by an on-device model that emphasizes privacy. The value from doing a total vertical integration with Apple silicon, an Apple device, an Apple AI foundation model, trained on Apple data is a distinct competitive advantage. 

If—and this is a big if—Apple’s products match the demos from today, this is a significant improvement in computing for more than 1 billion iPhone users. Of course, demos are by definition manicured videos, not actual products in action. After looking at hundreds of AI demos, my experience has taught me that executing these product rollouts at scale is a near-impossible challenge. The details are tricky and very hard to get right. Apple has numerous advantages—in custom silicon, resources, and install base—so maybe it’s the company to finally deliver. If it works, it is easy to picture Siri being extended to more powerful workflows on MacBooks.

For every instance of meta-layer AI improvement, there is a requisite decrease in the power of an individual software application. Siri will have more data and more access, and—worst of all for SaaS companies—it will be free. There are many apps in my life that I would love to never have to use again. Hopefully, soon, Siri will free me from them. At its most fundamental, every application becomes simply a store of data on top of which sits an LLM, generating insights and taking subsequent action. Instead of having a CRM, sales reps could record their phone calls, and AI could transcribe those calls into a database, pull out the necessary information to fill out the fields required, and schedule the next meeting—no application interface required. We move into a world where far less software requires a graphical user interface). We are still many years away from a world of “"headless everything," but you can see the foundations being laid.

There were also numerous updates that went beyond AI. 

Get me a big ol’ slice of platform pie (ecosystem evolution)

Apple devices feel like witchcraft to me when I jump from one to the other. Copying something on my iPhone only to seamlessly paste it onto my Mac makes me check my closet for black cats and toadstools. On that front, Apple continued to improve its ecosystem benefits—this time with a new password manager app.

The Passwords app is coming to MacOS, iOS, and VisionOS, the operating system powering the Apple Vision Pro headset. Password apps aren’t new, but Apple’s products differentiate by nailing the details. A centralized place to store passwords will be extremely useful.

More visually impressive platform updates came for the Vision Pro, which is adding the ability to turn 2-D photos into 3-D “spatial” photos. Emotionally visceral photos and videos are the killer capability for the Vision Pro, and I’m excited that Apple is making existing photos 3D. The company also improved the excellent Mac Virtual Display, which lets you grab your MacBook’s screen and make it an ultrawide display on the Vision Pro.

Individually, none of these upgrades justifies purchasing an Apple device. But when you add them together, the ecosystem makes it worth it for every device you own to be made by Apple. It isn’t quite the strong network effects enjoyed by platforms like Uber or Instagram, but there are strong amounts of ecosystem lock-in. 

Apple Intelligence

Despite the cutting-edge tech, Apple still trotted out age-old tropes. Capabilities that had been on Windows and Android for years announced to thunderous applause? Check. Window tiling—which allows windows to easily snap into place around your screen—is finally coming to Macs. Hilariously obtuse marketing that co-opts existing technology with sexy branding? Say hello to AI—branded as Apple Intelligence.

But despite my snark, this is a company that has played its cards correctly. LLMs are a portion of the tech stack that it mostly develops in-house, like the vast majority of AI used, according to early reports. It’s formed partnerships with leaders like ChatGPT for higher-risk queries that take place outside of the platform. 

Between the Microsoft, Google, and now Apple events, the technology landscape has…not changed all that much. It turns out that all of the advantages from the internet and mobile computing paradigm neatly transfer to artificial intelligence. Custom silicon for iPhones can be repurposed for running Siri queries. Data used for targeting advertisements can be used to customize workflows. None of these companies are sitting on their heels—they are all moving aggressively and spending billions. It is increasingly looking like LLMs will further entrench the existing tech giants. Maybe the new world is the exact same as the old, but 20 percent more expensive because now we have to pay for AI costs.