A familiar pattern happens on repeat at Every: A new model is released, and our token spend rises to new heights. This month’s arrival of Astra heralded double the token spend we saw in the age of Sol. “What are people doing using Astra that much, so we can figure out if it’s worth it?” asked Brandon Gell, our COO. As his head of operations, I couldn’t answer.
That gap in knowledge—is this token spend worth it?—is a problem increasing in size at the same rate that token spend is rising (fast!). It obscures one of the most important questions companies ask when spending: What did our money get us in return? Until now, model use, team output, and self-reported workflows have been directional clues to ROI, but they haven’t offered the granular insights most financial leaders want to see.
Models are smart enough that no one should settle for a blanket AWS-style bill anymore. Soon token spend won’t be a monolithic line item per lab, but broken apart based on the outcomes the work bought. An agent working as a personal assistant will be categorized as headcount. An MVP build will be categorized as R&D. Email automations will be categorized as SaaS.
Instead of endlessly burning through tokens, we’ll learn more sophisticated ways of estimating cost. For example, engineering teams will become fluent in sizing both time to delivery and tokens consumed in order to build. Assessing the ROI will go beyond pure output, and will include goals like faster learning, more capacity without increased headcount, or avoiding spending on unnecessary SaaS.
The industry is dipping its toe into new insights like these. OpenAI groups Work and Codex usage into broad use cases and tasks. Ramp integrations pull token spend insights into a dashboard for finance teams. These are meaningful first steps that transparently show what kind of work is consuming credits, but companies still need to connect that activity to a budget, an owner, and an outcome.
We’re doing our part. We’re building a skill that will allow our teammates to create their own AI spend P&Ls to share transparently, and our head of evals Mike Taylor is making personal benchmarks to help staff figure out which models can most capably handle their everyday work. This is what the future looks like: staying accountable while still supporting innovative work.
