Before the Deluge

Amiddle-aged woman speaks into an intercom outside a sleek, nondescript office building in San Francisco’s Mission Bay district.

“I’m here for lunch.”

“For lunch, ma’am?” A tinny voice crackles through the speaker. A tall, bearded security guard intercedes at the door before the conversation can go any further. “This is not a restaurant.”

Indeed, it is not. At 3180 18th Street, better known as OpenAI’s headquarters, you need an invitation just to enter the building. Once inside, you’re greeted in a lobby appointed like a slightly corporate five-star hotel, complete with accent books and lemon-wheeled water coolers.

OpenAI may not be a restaurant, but a few floors up, it is lunchtime. Employees all working toward the company’s stated mission of achieving Artificial General Intelligence (AGI) line up for harvest bowls, tacos, stir fry, and hot slices fresh out of what I’m told is San Francisco’s largest live-flame pizza oven.

As OpenAI competes with rival Anthropic to release new frontier models at an increasingly rapid clip, its engineers can ship code 10, even 100 times faster than before AI. Codex, the company’s coding agent and desktop app, extends that capacity beyond engineering; product managers, data scientists, researchers, and go-to-market employees use it to write, modify, debug, and submit code that runs on OpenAI’s internal systems. Many in the cafeteria are glued to their phones, perhaps texting, scrolling social media or, more likely, orchestrating agents through OpenAI’s mobile app.

It’s an exhilarating time for anyone building new products or tools. For the engineers responsible for maintaining the internal systems that make all this building possible, however, the feeling is closer to staying afloat.

We use ‘deluge’ a lot,” says Emma Tang, who heads the applied infrastructure data platform team. Tang’s group maintains the infrastructure that moves, analyzes, and secures the data flowing through the plumbing of every OpenAI product, including ChatGPT and Codex. The team’s volume of pull requests (PRs), or proposed changes to a shared codebase, has jumped five- to tenfold in the past year.

A meaningful share of that code arrives from people in non-technical parts of the organization. “We have users generating Flink jobs”—programs built with Apache Flink, a system that analyzes continuous streams of product data—“who don’t even know Flink,” Tang says. As the pace of change has increased, so have internal reliability problems, from delayed data processing to dashboards that fail to load. And when something breaks, the author often can’t help because they don’t fully understand what they’ve shipped.

Until recently, if someone in operations or on the go-to-market team wanted a custom internal app, they’d need to find an engineer to make it for them. Now, anyone at the company can “show up on Monday with a working prototype they built over the weekend,” Tang says. Her engineers act increasingly as educators and mediators, helping colleagues turn their AI-generated tools into reliable software. A vibe-coded app, for example, might unnecessarily refresh every few seconds, while the analytics system it pulls from updates only once a day. Making the data truly live means connecting it to a different production system, which raises questions largely invisible to people without a technical background—questions like, “Where will it run?,” “How should it retrieve information without slowing down the service everyone else relies on?,” or “Who will maintain it?”

The last question, in particular, often falls on Tang’s team. AI allows anyone at the company to build apps and systems that depend on the company’s shared data infrastructure, but should that code fail, her team inherits the problem.

“We become the catch-all,” Tang says. “Nobody owns it anymore except us.”

The applied infrastructure team’s first response to the deluge is also the tool that created it: Codex.

Within weeks of the release of GPT-5.5, adoption of the Codex desktop app within OpenAI reached nearly 100 percent. Engineers now run the bulk of their work from inside the app, reviewing AI-generated code, AI-generated tests (automated checks that confirm the code works as it should), and AI-proposed fixes. Because OpenAI employees have no token limits, an engineer can queue up a batch of UI bugs in Codex and have multiple agents independently write fixes, run them in test versions of the product, click through the affected flows in a browser, and flag resulting issues—all using the most powerful models (many employees are granted access to models before they’re available to the public), without thinking twice about cost or efficiency.

When Molly O’Connor joined OpenAI’s applied data platform team in October 2024, a typical day included getting paged for an incident, debugging it, and then moving on. Now, along with making the fix itself, she might also update the company’s shared Codex instructions so that it can correct similar problems in the future. O’Connor does the same when she spots recurring problems in pull requests—especially ones that could make the system less reliable.

Her role—to make sure OpenAI’s data infrastructure remains reliable and secure—hasn’t changed. But more of the work now involves turning what she knows about building and troubleshooting data pipelines, executing large-scale migrations, and testing and rolling out updates to data-processing systems into instructions Codex can follow.

James Katz, the OpenAI infrastructure engineer on call while Codex debugged the overnight export failure, is an extreme example of this evolution. Following the release of GPT-5.5, Katz’s team created, as an experiment, a skill that lets Codex query OpenAI’s internal data export system on behalf of approved users, without requiring them to know how to translate their requests into precisely formatted database instructions written in SQL. (“Holy crap, this is really working on the first shot most of the time,” Katz remembers thinking.) Within a couple of weeks, the new Codex skill accounted for more than half of the data export system’s total usage. Katz’s team was caught off guard; the tool’s immediate popularity was validating, but it also left him to deal with all of the edge cases and write guidance for the parts of the system no one had documented.

It’s been a balancing act, to say the least. Katz politely describes his team as “resource constrained”; any time devoted to the export skill is time not spent on existing infrastructure work. “There’s always more work on the backlog than we have people to support,” he says.

A similar dynamic is playing out throughout OpenAI. Technical staff member Bonnie Xu and head of realtime data infrastructure Aravind Suresh are on a small team that built and maintains OpenAI’s internal data analytics agent, which is used by more than 5,000 of the company’s employees to query its data warehouse.

Eight months ago, if someone wanted to modify the agent, they’d request the change via Slack. Six months ago, that morphed into “Hey, can I cut a PR?” Two months ago, it became, “I have a PR that Codex created for me. Can you review it?”

Pull requests now pour in from across the company. Like O’Connor and Katz, Xu and Suresh’s team manage the influx by creating shared Codex skills. By the time a PR reaches their team, Codex should have helped its author test, document, and refine it—which has gone a long way to filtering out the truly unsalvageable code changes.

Once a PR is ready, a separate, automated Codex bot reviews it, often catching things the first agent or author missed, Xu says. Human reviewers still look at every PR, with particular attention to changes that are large, risky, or crucial to the system’s reliability. For those, the team asks additional Codex agents to investigate specific concerns—for example, whether rewriting part of the code could disrupt another system or whether the change follows a project’s guidelines. The largest changes may go through the entire process again with more than a dozen subagents.

Codex isn’t a fix for everything. Tang’s team also built a Slack bot powered by a ChatGPT Workspace agent to divert the constant stream of questions from colleagues across the company—questions like “What permission do I need to update this table?” or “This data pipeline failed, can someone look?”

But even this patchwork of tools isn’t enough. “If anybody says they have a solution, they probably just don’t know the whole picture,” Tang says.

One thing is clear, however: Compared to what’s coming, today’s deluge will look like a drizzle.

We ain’t seen nothing yet,” says Venkat Venkataramani, OpenAI’s vice president of applied infrastructure engineering.

Dressed in a green-and-gray plaid flannel button-down, sneakers, and blue jeans, Venkataramani’s casual clothes and soft-spoken cadence belie his unique vantage point. As the person responsible for keeping OpenAI’s infrastructure running under the inundation of AI-generated code, he has a clearer view than almost anyone of what the company is facing—and what the rest of the software industry is about to face.

Today’s volume of AI generated code is unprecedented; in retrospect, says Venkataramani, it will look quaint. In the near future, a year’s worth of incidents could be compressed into a single week. “A flash flood is coming,” he says.

Unlike a flood that hits a city, the danger isn’t in the volume of the water alone. More high-quality code just means more useful—or at the very least, harmless—software. What’s dangerous is what the flood carries with it: bugs, security issues, and other problems hidden within the rush of proposed changes. At 10, 100, possibly 1,000 times the volume, even a small proportion of contaminated code becomes a serious threat.

The first line of defense is continuous integration (CI), the automated checkpoint that tests new code before it merges into a shared codebase. Farther along is continuous deployment (CD), which ships approved code into production. Built in a pre-agentic world, both were designed for code written at a human pace. But as the cost of generating code falls toward zero and volume explodes, the number of changes waiting to be tested and merged will overwhelm the system. Problems CI fails to catch pass through CD into production, where failures become incidents.

The solution, in Venkataramani’s view, isn’t simply to expand the existing system’s capacity. “If you just scale the CI infra and the CD infra, you’re just going to have 100x more code going in every day,” he says. “And then what?” Scaling CI and CD moves more code through the system without making that code safer.

His solution resembles modern stormwater management. Instead of relying on a single downstream checkpoint, it filters for different problems at multiple points. “You add 10 such filters,” he says, “you’re going to eventually have clean water.”

The first filter is the most intuitive, and what people like O’Connor are already doing: Shape the code as it’s written, so a team’s standards are built into how a tool like Codex writes and reviews it.

Skills provide one layer of protection, but Tang wants something more enforceable: stronger checks that live in Codex, in the code-review system, or in agents supplied by the teams responsible for the affected systems. In the latter scenario, one agent writes the code while another applies the owning team’s expertise to assess whether the proposed change is appropriate for that system.

These review agents could be specialized by domain—perhaps one for inspecting code that processes large volumes of information, another for auditing real-time data pipelines, and so forth—that are distinct from the agent writing the code. That way, one agent can build whatever a user asked for while review agents can fully focus on making sure that code doesn’t break anything.

Once the code is running, a different class of agents could monitor the systems around the clock for live reliability problems. Rather than reviewing code, these agents would watch how it behaves and, Tang hopes, catch and isolate problems before they affect other systems.

Her idea for a final line of defense is the most speculative, and goes one step further. Whereas always-on agents would monitor systems and respond when they detect trouble, Tang envisions a near-future in which judgment is baked into the underlying infrastructure itself. In this scenario, technical systems could autonomously refuse to execute an unsafe action, even if every previous guardrail approved it. The infrastructure would have its own “level of intelligence, almost like a human operator sitting in that service,” Tang says.

If all this comes to pass, OpenAI will still need engineers. But their days will no longer involve writing code or identifying and patching bugs—AI will do that for them.

Tang’s team is already living an early version of this reality. “People who used to be the bricklayers, where every piece of code is handwritten,” she says, now spend their days as construction managers. “They have like 10 Codex [threads] and they’re just monitoring whether it’s doing the right thing.”

As the models improve, Venkataramani expects code review to give way to prompt review, and then plan review. Engineers will no longer review lines of code and instead evaluate the intentions behind them, starting with specifications, and then architecture documents, and eventually the business problem itself. New coding environments might return only a high-level explanation of what the code does, he says. A complex project that today requires millions of lines of code could be expressed as a short set of instructions laying out what the system should do, with the model filling in the rest.

This, unsurprisingly, changes the job requirement. Instead of solid executors, you need people who can answer messy, ambiguous questions that don’t have clear-cut answers, like how to determine whether to fund a project or the best way to measure whether an initiative is paying off. OpenAI has long hired people who are “very entrepreneurial, very self-directed, who have their own opinion and really strong ownership,” Tang says, but these qualities have calcified into non-negotiables as the models absorb more of the operational work. “We only hire those people now.”

To Venkataramani, on the other side of today’s struggle to keep up with AI-generated code are systems that will make even our largest software projects seem minuscule by comparison. “We will look back at the days where the most complex systems were Spark and Linux as the first set of five-story and 10-story buildings built in Chicago,” he says. “They were considered skyscrapers. These artisanal buildings.”

Laura Entis is a staff writer at Every. You can follow her on LinkedIn.

Produced and designed by Douglas Brundage, Jack Cheng, and Daniel Rodrigues.