Tracarbon: Track GPU power and carbon emissions while running local LLMs

As local language models grow larger and developers run them on increasingly powerful hardware, the energy cost of inference has become a measurable concern. Tracarbon, an open-source Python library, gives developers a way to track exactly how much power their GPU and CPU consume during a run and estimate the carbon emissions that result.

What Tracarbon measures and how

Tracarbon detects your device and location automatically, then collects energy consumption samples at a configurable interval. On Mac, it reads Apple Silicon CPU, GPU, memory, and Neural Engine power through IOReport without requiring elevated permissions. On Linux, it uses the RAPL interface for Intel processors and the powercap or amd_energy driver for AMD chips on kernel 5.8 or later. On Windows, it pulls NVIDIA GPU power via nvidia-smi, though CPU and memory measurements are not available on that platform.

The library exports its results through pluggable exporters. Stdout prints metrics directly to the terminal, JSON writes them to a file, Prometheus sends them to a Prometheus server, and Datadog forwards them to Datadog's monitoring platform. The Prometheus exporter includes container-level metrics for Kubernetes deployments, breaking down energy consumption and carbon emissions by pod, namespace, and container.

Carbon intensity, the grams of CO2 equivalent per kilowatt-hour, comes from the CO2Signal API for non-commercial use or ElectricityMaps for commercial use. Both provide near real-time data on the carbon intensity of electricity in your region. If you do not have an API key, Tracarbon loads carbon intensity statistically from a bundled file, which is less accurate but functional for initial testing.

GPU support across vendors

The library tracks power consumption for NVIDIA GPUs through nvidia-smi on Linux, Windows, and Intel Macs, with support for multi-GPU setups. AMD GPUs are supported through rocm-smi or amd-smi on Linux, also with multi-GPU support. Apple Silicon's integrated GPU gets measured through the same IOReport mechanism as the CPU, with a fallback to powermetrics that requires sudo. Intel discrete GPUs are not yet supported.

For cloud deployments, Tracarbon uses hardware usage data combined with carbon emissions datasets from the cloud-carbon-coefficients project. It supports AWS, GCP, and Azure, mapping instance types to their grid emissions factors to estimate the carbon cost of running workloads in each region.

Kubernetes and container awareness

When deployed in Kubernetes, Tracarbon runs per node and scopes container metrics to that node using the NODE_NAME environment variable set through the Downward API. The Prometheus exporter exposes metrics with labels for pod name, pod namespace, container name, platform, location, and source, making it possible to attribute energy consumption and emissions to specific workloads in a cluster.

If Kubernetes returns no pod metrics, the CLI logs a warning but continues exporting host-level metrics. Zero values are still exported, which keeps metric series continuous for dashboards and alerting.

Why this matters for local LLM development

Running large language models locally has become increasingly common as hardware improves and open-weight models proliferate. A developer fine-tuning a 70B parameter model on an NVIDIA A100 draws hundreds of watts for hours at a time. Without measurement, that energy cost is invisible. Tracarbon makes it visible, which is the first step toward making it actionable.

The library ties into a growing body of research on sustainable machine learning. Academic work from 2023 through 2025 has examined carbon emissions tracking for ML models, analyzed eco-feedback tools through the lens of sustainable HCI, and studied how model selection decisions affect overall environmental impact. Tracarbon gives developers a concrete tool to act on that research in their own workflows.

For teams running inference at scale, whether on local hardware or in the cloud, the ability to measure and export energy and emissions data integrates naturally into existing monitoring stacks. Prometheus and Datadog exporters mean the data shows up alongside other infrastructure metrics, not in a separate silo.

Getting started

Installation is a single pip command, with optional extras for the Prometheus, Datadog, and Kubernetes exporters. Running tracarbon run from the command line collects measurements until you stop it, then prints the total CO2 grams emitted. The Python API wraps the same functionality, letting you instrument a script or application by starting and stopping a Tracarbon instance around the code you want to measure.

Configuration happens through environment variables or a .env file, covering the API key for carbon intensity data, the metric prefix for exported names, the sampling interval, and the Kubernetes node name. The library defaults to sensible values for each of these, so you can get meaningful results with no configuration at all on supported hardware.

The project is MIT licensed and available on GitHub, with documentation covering setup, exporters, and the full list of supported devices and cloud providers.