Skip to main content

OpenCost

OpenCost is a CNCF project for real-time cost monitoring of Kubernetes workloads. It breaks down the cost of CPU, memory, GPU, storage and network per namespace, workload and label, and exposes the data through an API, a UI and Prometheus metrics.

Accessing the UI​

OpenCost has no built-in authentication. We therefore expose it behind an oauth2-proxy sidecar which authenticates against Keycloak.

The same opencost Service also exposes the cost-model API and /metrics on port 9003 (forward that instead if you want to query the API directly).

A datasource is required — and we run a dedicated one​

OpenCost needs a Prometheus-compatible datasource. It does not just read metrics: it also writes its own pricing metrics (node_total_hourly_cost, container_cpu_allocation, pv_hourly_cost, …) to its /metrics endpoint and then queries them back to compute allocations.

Haven+ ships the full LGTM stack, so the obvious idea is to point OpenCost at Mimir. Mimir is a valid OpenCost backend on paper (it speaks the Prometheus query API and the chart supports the X-Scope-OrgID tenancy header), but we deliberately do not do this. Instead, each cluster runs a small Prometheus scoped to the opencost namespace.

Why not reuse Mimir/LGTM​

The deciding factor is the self-metrics round-trip described above:

  • With a local Prometheus, OpenCost writes a metric and reads it back within the same cluster in seconds.
  • Through Mimir, that same loop becomes: Alloy scrapes OpenCost → remote_write → ingester → query-frontend → OpenCost reads it back. That round-trip adds latency and staleness/gaps inside OpenCost's hot path, which is the classic "write to a remote store and immediately read your own writes" anti-pattern. Cost numbers become unreliable.

Two secondary reasons reinforce this:

  • Scrape invariant. Reusing Mimir would require guaranteeing that Alloy scrapes cadvisor, kube-state-metrics and the OpenCost pod itself into the one tenant OpenCost queries, on every cluster. That invariant is easy to break and hard to notice.
  • Noisy neighbour. OpenCost fires heavy range queries every cycle; running them against the shared Mimir that also serves Grafana adds avoidable load.

A dedicated, namespaced Prometheus sidesteps all three.

Cloud pricing and provider RBAC​

Out of the box OpenCost uses its on-prem default pricing (configurable per-resource rates), which works on every provider without credentials.

To get real cloud rates (cloud bills, spot pricing, rate cards), OpenCost integrates with each cloud provider's billing API, and that integration needs provider-specific credentials and RBAC. Rather than reproduce it here, follow the official OpenCost guides:

Supply the resulting credential as a Sealed Secret and wire it into the OpenCost overlay for that cluster.