Organizations running AI workloads face a billing problem that is less about the technology and more about the accounting. The same GPU that serves one user's retrieval-heavy query might also serve another user's lightweight chat, and the cost data arrives through completely disconnected channels: Kubernetes resource allocations from self-hosted inference clusters, gateway logs from the routing layer, and per-token invoices from cloud API providers like OpenAI and Anthropic. Without a way to tie these threads together, organizations cannot answer a deceptively simple question: who actually pays for what?
Who Pays for the KV Cache? Attributing Shared AI Inference Spend Across Kubernetes and LLM Provider Bills, by a team presenting on September 21, 2026, tackles this problem head-on. The paper introduces unalloc, an open-source tool that joins OpenCost allocation data, LiteLLM gateway logs, and per-token billing records from OpenAI and Anthropic into a single unified ledger, then uses that ledger to study exactly where attribution breaks down at the seams between these systems.
The fragmented cost landscape
The problem begins with the fact that AI inference costs flow through at least three independent accounting channels. Kubernetes allocations tracked by OpenCost assign GPU and memory resources to pods based on resource requests and node pricing, but these allocations struggle to account for shared infrastructure components like the KV cache store, the inference gateway, and the scheduler. Gateway logs from LiteLLM record individual token counts and request metadata but do not know the underlying GPU costs. Per-token bills from API providers are opaque: they tell you the total charge but not how that charge maps to specific workloads, tenants, or inference sessions running on your own infrastructure.
When these systems are reconciled manually, the seams between them produce significant attribution errors. The paper quantifies this through five case studies that range from simulated inference serving to real deployments, each designed to expose a specific failure mode.
Case studies: where attribution breaks
The first set of findings concerns a constructed multi-pod deployment scenario using one month of synthetic OpenCost allocation data. The scenario is designed to test what happens when owner labels, which map GPU spend to organizational units or teams, are set only on the leader pods of a LeaderWorkerSet deployment pattern. The result is striking: 66% of that deployment's GPU bill goes unowned, with no label to attribute it to. The natural fallback mechanism, which assigns unattributed costs based on the Helm chart name, dumps 61% of the bill into a single Helm chart bucket while the headline unallocated share drops to a deceptively low 4%. This illustrates a common trap: the unallocated percentage looks small when measured against the total, but the distribution is wildly skewed, with one Helm chart name absorbing the vast majority of orphaned spend.
The second finding reveals that enabling every available data source does not guarantee accuracy; it can introduce double counting. When all cost sources are combined without deduplication, every gateway transaction gets counted twice, inflating the total spend and making it impossible to identify where money is actually going. The third finding is more subtle: reading a single page of a billing API report captures only a quarter of the total spend, suggesting that API-level billing data is inherently partial and cannot serve as a complete cost picture on its own.
Inside the shared inference server: metering rules decide who pays
The second half of the paper shifts from the seams between systems to the interior of a shared inference server. On an NVIDIA H100 running vLLM, the choice of metering rule becomes the decisive factor in how costs are attributed to tenants. The authors compare two approaches: a token meter, which assigns costs proportional to the number of tokens processed, and an equal time-share meter, which splits costs evenly across tenants regardless of usage.
The difference is substantial. For a retrieval-heavy tenant, the token meter assigns 12 to 14 percentage points more of the total bill than the equal time-share meter at every load level tested. This is because retrieval-heavy workloads process more tokens per request, and a token-based accounting reflects that reality while a flat time-share does not.
The paper also reports that GPU utilization reads 97 to 99% across configured loads ranging from 2 to 16 requests per second, which corresponds to 3.7 to 26.9 completed requests per second, with the configured rate counting only session-initial arrivals rather than all completed requests. Power draw tracks the load closely, confirming that energy consumption is a reliable proxy for actual compute usage in this setting. However, the authors are careful to note that neither meter represents ground truth: the token meter may over-penalize tenants with long-context queries that are cache-assisted, while the time-share meter may undercharge tenants whose workloads are compute-intensive per token.
Positioning against Shapley-based attribution
The paper frames its findings against recent work on Shapley-based energy attribution, which uses cooperative game theory to distribute costs among tenants based on their marginal contribution to total resource usage. Shapley values provide a theoretically fair allocation but are computationally expensive to compute in practice, especially in dynamic serving environments with thousands of concurrent requests. The authors position their results as a practical alternative that trades some theoretical elegance for operational tractability, arguing that even a simple token meter captures most of the variance that Shapley-based methods would identify.
What this means in practice
For ML platform teams and FinOps practitioners, the paper delivers a clear operational message: the choice of attribution mechanism matters as much as the raw cost of compute. A 12 to 14 percentage point difference in cost allocation between metering strategies translates directly into internal billing disputes, misaligned engineering incentives, and incorrect ROI calculations for AI workloads.
The unalloc tool and the open-source repository, which includes code, raw data, captured evidence, figures, and a regeneration script for the paper's results, provide a practical starting point for organizations to audit their own cost attribution. The paper's finding that owner labels set on leader pods alone leave two-thirds of GPU spend unowned is a specific, actionable diagnosis that any Kubernetes-deployed LLM serving system should check immediately.
The broader implication is that the financial plumbing of AI inference is still immature. As organizations move from experimental deployments to production-scale serving across Kubernetes clusters and cloud APIs, the lack of standardized, cross-system cost attribution will become an increasingly pressing operational problem. This paper does not solve it entirely, but it identifies precisely where the breakdowns occur and provides the tools to measure them.