We Caught an AI Cheating by Watching It Play ‘StarCraft’

Was this newsletter forwarded to you? Sign up to get it in your inbox.

AI labs call agents “persistent” when they can carry their progress forward across sessions to keep working toward a goal.They can spend days attacking a problem from multiple angles, testing approaches and changing tactics when they fail. This persistence has produced breakthroughs in mathematics and physics. Sometimes, though, the “different approach” involves breaking the rules.

In July, OpenAI agents being tested on cybersecurity broke into Hugging Face, a platform for sharing AI models and datasets, while looking for test answers. OpenAI’s later investigation found that agents had been getting around internet restrictions since May. Some of that activity had been noticed internally, but the company didn’t connect its agents to the attack until nine days after they began executing code on Hugging Face’s servers.

Earlier this month, an agent running GPT-6 Astra also went looking for answers outside its test environment, during a 48-hour contest we ran at Good Start Labs, where we study and improve AI models through games. We’d partnered with researcher Kai McPheeters to create StarSkirmish, a platform where models write bots to play the 1999 strategy game StarCraft: Brood War.

Unlike the OpenAI incident, though, we noticed the cheating in minutes. Watching the game made a sudden change in the bot’s behavior visible—and prompted us to check what the agent had done. We’d originally set out to see whether frontier AI models could write bots driven by complex algorithms that beat the best human-built ones. The contest showed why judging an agent by its results alone isn’t enough; we also need to see how it gets them.

$400+ in credits and discounts for your next AI build

Join the free Nebius AI Builder Program for $400+ in credits and discounts from Nebius, LangChain, Tavily, Toloka, and more. Plus, working code, free courses, and expert office hours. Choose your models and tools and build alongside a growing community of AI builders.

Join Nebius AI Builder Program today

Want to sponsor Every? Click here.