Agents will soon be responsible for most economic output, and they will keep making mistakes—some expensive and some catastrophic. As production data is deleted and critical security vulnerabilities are introduced, we’re already seeing glimpses of how costly problems with agents can be.
I used to think better models would fix this. But as agents get more capable, they also get more complex—harder to test, with more ways to fail. At the same time, they’re trusted with higher-stakes work like legal advice and prescribing medication.
There’s simply no comprehensive set of evals that will capture every edge case, so as agents run longer and are given more open-ended tools, the real world becomes the test. Higher stakes mean less room for mistakes.
Someone has to track these mistakes, and for now, that’s us. They’re problems humans can still solve.
Our purpose as humans has largely come from solving problems, but there’s a shrinking gap between what makes us special and what agents are capable of. Growing up, my mother was one of the best translators in the world, a job that requires intelligence, skill, and grit. In that way, it's very human. Translation was one of the first careers to die out to automation, but it won't be the last.
In many ways, it feels hard to believe. There are still so many things humans are uniquely capable of, like starting a company, writing a great book, or perfectly architecting a production-scale backend. Each of those is a problem agents haven’t solved yet.
I define a problem as any case where a human can still improve an agent’s output (e.g., where a human has a better sense of what “good” looks like). Someday, agents will have no problems left, but until that day, checking AI’s work will be humanity’s last problem.
