As enterprises move AI from experimentation into production, a quiet but costly problem is surfacing: nobody can fully account for where the tokens are going. Most organizations struggle with basic visibility into how many tokens a team consumed and against which model, since the data is scattered across vendor consoles, and tying that consumption back to a team, project, and budget for management and chargeback purposes adds another layer of difficulty. Compounding the issue, production environments typically involve multiple providers, each bringing its own key, bill, and audit trail, causing the organization's spending to fragment along financial lines. It's a problem familiar to anyone who has watched cloud chargeback models evolve over the past decade, but AI introduces a new wrinkle in both speed and scale.
VMblog recently spoke with Matthew Shaxted, CEO of Parallel Works, about how his company's roots in high-performance computing and government infrastructure are shaping a different approach to AI governance. Drawing on years spent governing compute and storage for national labs, defense programs, and large system integrators, Shaxted explains why his team's instinct was to treat tokens as just one more metered resource on the same ledger that already tracks core-hours and storage, rather than building an entirely new governance category from scratch. In this Q&A, Shaxted discusses why token budgeting is on a collision course with traditional compute chargeback, how hybrid public-private AI deployments are reshaping enterprise security and control, and why he believes the operating model—not raw model capability—will ultimately separate the organizations that scale AI successfully from those left explaining a rising bill with no clear return.
VMblog: Parallel Works has deep roots in HPC and government computing, including production deployments inside large system-integrator and defense environments. How does that heritage in governing compute and storage at scale shape the way you approached AI governance differently than vendors coming purely from the SaaS or developer-tools side?
Matthew Shaxted: Our starting point was a bit different, because the problem was not new to us. We have spent years governing compute and storage for national labs, defense programs, and large system integrators, where you cannot just hand people unlimited capacity. You give a team an allocation, you meter what they use against it, and you charge it back. That is the HPC operating model, and it is decades old. So when AI arrived, our instinct was not to treat it as a brand-new category that needed a brand-new tool. It was to treat tokens as one more metered resource on the same ledger that already records core-hours and storage. Vendors coming purely from the SaaS or developer-tools side tend to start at the API and add budgets on top, which works for token spend in isolation but has no view of the cluster underneath. Because we own the compute layer, the allocation that caps a team's frontier-model tokens is the same one that governs its GPU time, and that comes directly from where we started. The other thing the government work taught us is that governance has to sit in the request path, not in a monthly report, the same way a scheduler checks a job against its allocation before it runs rather than after.
VMblog: Token budgets and chargebacks are ultimately a financial accountability mechanism — and any time you introduce caps or visibility into how individuals or teams use a resource, there's potential for friction or pushback. What has adoption looked like in practice, and what's your advice to IT leaders trying to roll out AI governance without it feeling like a crackdown on innovation?
Shaxted: There is real potential for friction, and in our experience it usually comes down to how you introduce it. If you start with the cap, you tend to get pushback. What has worked better is starting with visibility. Most of the resistance we hear is less about the limits themselves and more about surprise, a team finding out after the fact that it ran through something it could not even see. So we usually suggest beginning with showback, giving every team a clear view of its own consumption before you put a number on it. Once people can see their usage, a budget tends to feel less like a crackdown and more like a normal operating constraint, in the same way an engineer accepts that a cluster has a finite number of nodes. My advice to IT leaders is to sequence it that way deliberately: turn on metering and showback first, set initial budgets with real headroom rather than tight ceilings, and make the whole thing self-service so a team can see where it stands without filing a ticket. The way I would frame it is that governance is what lets you say yes more often. It is the control that makes it safe to open frontier models to more of the organization, rather than a way of taking them away. When people understand that the budget is what keeps a surprise bill from shutting off their access, the conversation gets a lot easier.
VMblog: Many organizations rushed to deploy AI over the past two years. What governance and operational challenges are now emerging as enterprises move from experimentation to large-scale production AI?
Shaxted: The challenges that surface in production are mostly the ones that did not matter in a pilot. In experimentation there is no real contention, so nobody asks who is using which model or what it costs. In production, those questions tend to arrive all at once. The first is simple visibility. Most organizations genuinely cannot tell you how many tokens a given team consumed last week, or against which model, because the data is scattered across vendor consoles. The second is attribution, tying that consumption back to a team, a project, and a budget so it can be managed and charged back. The third is enforcement, being able to stop an overrun while there is still a budget to defend rather than discovering it on the invoice. On top of that, production usually means more than one provider, and each commercial API brings its own key, bill, and audit trail, so the organization fragments along financial lines. If any of those models are private and self-hosted, you get a second kind of fragmentation, because now teams contend for the same GPUs and someone has to arbitrate fair share. None of this is exotic; it is the ordinary operational overhead of running a shared resource at scale. I think it caught a lot of teams off guard because they had been treating AI as a feature, when in production it behaves more like infrastructure.
VMblog: Token consumption has become a major cost driver for AI initiatives. Do you see AI budgeting and chargeback models becoming as important as traditional compute and storage chargeback models in enterprise IT? Why?
Shaxted: Yes, and I would go a step further. I think AI chargeback and compute chargeback stop being two separate disciplines and become one. The reason traditional compute and storage chargeback exists is that those are shared, finite, expensive resources that multiple teams draw on, so you have to attribute the cost to whoever incurred it. Tokens now fit that description: a shared, metered, and increasingly large line item, growing faster than the compute around them. What makes AI different, and in some ways more urgent, is that the spend happens in milliseconds and agents spend it in loops, so a misconfigured workload can run up a number overnight that a human-paced process never would have reached. The mistake I would caution against is standing up a separate, token-only chargeback system next to the compute one. The teams running agents are usually the same teams running training jobs and storage on the same infrastructure, so what they actually need is a single statement that shows tokens, GPU-hours, and storage together, attributed to the same budgets. Two parallel chargeback systems that never reconcile add accounting overhead without giving you a single, governable view of the spend.
VMblog: Organizations are increasingly using a mix of public AI services and privately hosted models. How is this hybrid AI approach changing the way enterprises think about infrastructure management, security, and governance?
Shaxted: Hybrid is quickly becoming the default rather than a transitional state, and it changes the questions enterprises ask. On the management side, the moment you are running public APIs and private models at the same time, you cannot treat them as two separate worlds with two separate sets of keys, budgets, and audit trails. You need one control plane that presents both behind a single endpoint, so a workload can move between a frontier API and a self-hosted model without re-onboarding the user or rebuilding the integration. On security, the private side is usually the main motivation. Teams stand up open-weight models on infrastructure they control precisely so that sensitive or regulated data, and the model weights themselves, never leave the boundary. That is the pattern that makes AI usable in environments where sending data to a public API is simply not allowed, which is a lot of the government and defense work we do, where the platform runs on-premises, in air-gapped settings, and under an IL5, FedRAMP High aligned authorization. On governance, the important shift is that the policy has to span both. The same allocation that limits a team's public-API tokens should govern its share of a private deployment, because otherwise you have governed half the estate and left the other half open. I think hybrid is a big part of why the gateway layer needs to exist, since it is the natural place to see both halves at once.
VMblog: There's been significant focus on GPUs as the foundation of AI infrastructure. Do you believe the industry is overlooking the operational and financial governance layer required to make AI sustainable at scale?
Shaxted: I do, and it is the throughline of everything we have said since the launch. The GPU is necessary, but owning accelerators is a procurement exercise, while making them productive and accountable is an operating-model problem, and the industry has spent far more attention on the first than the second. We watched this same movie one layer down. Organizations bought GPUs faster than they could put them to work, and capital stranded as idle hardware because there was no operating model to schedule and attribute it. Tokens show the same underlying problem with the opposite symptom. This time the capital is not sitting idle, it is being consumed quickly, and what gets stranded is the ability to say which spend produced value and to stop the spend that did not. In both cases the missing piece is the same governance and accounting layer between the purchase and the outcome. So yes, I think that layer is underbuilt relative to the hardware, and the gap is part of what decides whether an AI program is sustainable. Sustainability comes less from buying more GPUs and more from governing how the capacity you already have gets consumed.
VMblog: Looking ahead over the next three to five years, what do you think will differentiate organizations that successfully scale AI from those that struggle to realize business value from their AI investments?
Shaxted: I think the differentiator will be the operating model, not the models themselves. Model capability is improving quickly and commoditizing at the same time, so a few years out, access to a good model will not be what sets organizations apart. The ones that scale successfully will be the ones that built the discipline around it. They meter consumption where it happens, attribute it to teams and budgets, and, just as importantly, they do the unglamorous work of tying that spend to a defined outcome and measuring whether it paid off. The research community is actually ahead of the enterprise here, in how rigorously places like the national labs hold large infrastructure investments to a measurable result. The organizations that struggle will be the ones that scaled adoption without ever building the layer that connects spend to value, and they will end up where Uber's COO did, with a rising bill and no clear answer to what they got for it. So I think the next few years of AI get shaped as much by economics and governance as by raw model performance. The capability itself is going to be broadly available, so the differentiation moves to who can deploy it accountably, run it economically across public and private models, and prove the return.