Upgrades and backups

KCM updates its database schema at startup, so most upgrades are a new image and a restart. This page lists what to back up first and the few versions that ask for an action.

What to back up

Two things together, always as a pair:

  1. The PostgreSQL database: clusters, users, roles, audit trail, templates, test scenarios and the encrypted secrets.
    # Docker Compose
    docker exec kcm-postgres pg_dump -U kcm_user -d kcm_db -Fc > kcm_db-$(date +%F).dump
  2. The encryption key pair that protects the stored secrets (SASL and truststore passwords, ksqlDB, Schema Registry and Kafka Connect credentials, SSO client secrets, Git and Confluent Cloud credentials):
    • Docker Compose: the folder 00_kcm_with_docker_compose/data/keys (both jwt/ and encryption/);
    • Helm: the Secret kcm-api-encryption-keys (distinct pair), or kcm-api-keys when the encryption pair is the JWT pair of an older install.
A database without its key is half a backup

KCM still starts with another key pair, but every stored secret becomes unreadable and must be entered again. Keep the key pair in a safe place, separate from the database dump, and never share it.

Also keep your docker-compose.yml or Helm values file, and your license file if you installed one from a file.

How to upgrade

Docker Compose

cd KCM && git pull
cd 00_kcm_with_docker_compose
docker compose pull
docker compose up -d
docker logs -f kcm-api

Helm

helm upgrade kcm oci://registry-1.docker.io/kafkaetech/kcm --version 1.5.1 -n kcm -f values.yaml

Keep the same values from one upgrade to the next, in particular api.encryptionKeys. The chart refuses an upgrade that would change the encryption key of a running install, unless you explicitly allow it.

Before you upgrade
  • Back up the database and the key pair.
  • Deploy kcm-api and kcm-ui with the same version.
  • Read the notes of every version between yours and the target, below and in the release notes.

Encryption key pair on Helm

Up to chart 1.0.22 the encryption pair was the JWT pair. Since then the chart can mount a distinct pair from its own Secret (api.encryptionKeys):

Change the encryption pair

To move to a distinct pair, or to replace a pair that may have leaked, keep the old private key mounted while the API re-encrypts the stored secrets at startup:

  1. Create the new pair and its Secret kcm-api-encryption-keys.
  2. Upgrade with the new pair, and the old private key as the previous key:
    helm upgrade kcm oci://registry-1.docker.io/kafkaetech/kcm --version 1.5.1 -n kcm --reuse-values \
      --set api.encryptionKeys.enabled=true \
      --set api.env.KCM_ENCRYPT_PREVIOUS_PRIVATE_KEY=/app/keys/private.pem
  3. Check the API log: Key change: N secret(s) re-encrypted with the current key, and no line saying that secrets are encrypted with another key.
  4. At the next upgrade, clear the previous key: --set api.env.KCM_ENCRYPT_PREVIOUS_PRIVATE_KEY="".

If the old pair may have been read by someone else, also change the secrets themselves at their source (Kafka SASL passwords, Schema Registry, Kafka Connect and ksqlDB credentials, SSO client secrets, Git and Confluent Cloud credentials).

Version notes

Only the versions that ask for an action, or change a behaviour you may rely on, are listed. Every other upgrade needs no configuration change.

To chart 1.5.1

To 1.5.0

To 1.4.x

To 1.3.x

To 1.2.x

Regenerate the Compose keys (from 1.2.0 or earlier)

Up to 1.2.0 the Compose repository shipped a sample key pair. Consider it compromised: from 1.2.1 the API refuses to start with it as its session signing pair.

  1. Update the repository (git pull) and leave your old data/keys/private.pem in place for now.
  2. docker compose pull && docker compose up -d. The API generates two new pairs in data/keys/jwt and data/keys/encryption, and re-encrypts every stored secret with the new encryption pair. Every user signs in again.
  3. Check the API log (docker logs kcm-api): no line saying that secrets are encrypted with another key.
  4. Delete the old data/keys/private.pem and data/keys/public.pem.
  5. Change the secrets stored in KCM at their source, then enter them again in KCM.
  6. Change the admin password if it is still the default one.

To 1.1.x

Within 1.0.x