Sealed Secrets

Kubernetes Secrets are managed using the tool Sealed Secrets. Sealed Secrets allows you to define your Secrets locally, encrypt them and turn them into a SealedSecret resource. A SealedSecret resource is safe to store in any (public) repository. A deployed SealedSecret resource can be decrypted only by a controller running in the cluster and nobody else (not even the original author) is able to obtain the original Secret from the SealedSecret.

Prerequisites

To be able to use Sealed Secrets, it is required to install kubeseal.

Kubeseal is a local client running on your host that you will use to encrypt any Kubernetes Secrets locally and turn them into SealedSecret resources. If you are using VSCode, it is highly adviced to install the Kubeseal extension to ease operations.

Usage

Sealed Secrets employs asymmetric encryption. In this reference repository, we use a static (private) key to ensure that our secrets persist across the lifecycle of a (local) cluster. Using the key generated by the Sealed Secrets controller would render our Sealed Secrets unusable if a new cluster with a Sealed Secrets controller is created.

Important Note for "Real Environments", especially production environments

In environments outside this reference repository, especially production environments, never use the static-key.yaml. Always exclude it and allow the Sealed Secrets controller to generate a key automatically.

The static key is located in infrastructure/sealed-secrets/overlays/local/static-key/static-key.yaml. In the same folder the public portion is included as well. You can use this key to generate a Sealed Secret resource:

kubeseal --cert infrastructure/sealed-secrets/overlays/local/static-key/pub.key --controller-namespace sealed-secrets -n default -f secret.yaml -w sealed.yaml

This command outputs a Sealed Secret resource based on your secret.yaml in a file named sealed.yaml.

Generating new static keys

For any other overlays, you could use the same static key by just copying the static-key folder to the relevant overlay and referring to it in the kustomization.yaml. If for any reason you need to generate a new static key, or use a dedicated static key for an overlay, you can follow the next steps:

Export some variables:

export PRIVATEKEY="priv.key"
export PUBLICKEY="pub.key"
export NAMESPACE="sealed-secrets"
export SECRETNAME="static-key"

Generate keys:

openssl req -x509 -days 365 -nodes -newkey rsa:4096 -keyout "$PRIVATEKEY" -out "$PUBLICKEY" -subj "/CN=sealed-secret/O=sealed-secret"

Write the keys to a Kubernetes secret:

kubectl -n "$NAMESPACE" create secret tls "$SECRETNAME" --cert="$PUBLICKEY" --key="$PRIVATEKEY" --dry-run=client -o yaml | yq e '.metadata.labels["sealedsecrets.bitnami.com/sealed-secrets-key"] = "active"' - > static-key.yaml && rm priv.key

Use the static-key.yaml as resource in the relevant overlay / kustomization.yaml. The logs of the Sealed Secrets controller should display that you are using the static key:

time=2025-04-01T14:44:07.065Z level=INFO msg="registered private key" secretname=static-key

Ensure that you also copy the generated pub.key in the repo, as this key should be used to encrypt secrets by generating Sealed Secrets resources.