Upgrades and backups
KCM updates its database schema at startup, so most upgrades are a new image and a restart. This page lists what to back up first and the few versions that ask for an action.
What to back up
Two things together, always as a pair:
- The PostgreSQL database: clusters, users, roles, audit trail, templates, test scenarios and
the encrypted secrets.
# Docker Compose docker exec kcm-postgres pg_dump -U kcm_user -d kcm_db -Fc > kcm_db-$(date +%F).dump - The encryption key pair that protects the stored secrets (SASL and truststore passwords,
ksqlDB, Schema Registry and Kafka Connect credentials, SSO client secrets, Git and Confluent Cloud
credentials):
- Docker Compose: the folder
00_kcm_with_docker_compose/data/keys(bothjwt/andencryption/); - Helm: the Secret
kcm-api-encryption-keys(distinct pair), orkcm-api-keyswhen the encryption pair is the JWT pair of an older install.
- Docker Compose: the folder
KCM still starts with another key pair, but every stored secret becomes unreadable and must be entered again. Keep the key pair in a safe place, separate from the database dump, and never share it.
Also keep your docker-compose.yml or Helm values file, and your license file if you installed one
from a file.
How to upgrade
Docker Compose
cd KCM && git pull
cd 00_kcm_with_docker_compose
docker compose pull
docker compose up -d
docker logs -f kcm-api
Helm
helm upgrade kcm oci://registry-1.docker.io/kafkaetech/kcm --version 1.5.1 -n kcm -f values.yaml
Keep the same values from one upgrade to the next, in particular api.encryptionKeys. The chart
refuses an upgrade that would change the encryption key of a running install, unless you explicitly allow it.
- Back up the database and the key pair.
- Deploy
kcm-apiandkcm-uiwith the same version. - Read the notes of every version between yours and the target, below and in the release notes.
Encryption key pair on Helm
Up to chart 1.0.22 the encryption pair was the JWT pair. Since then the chart can mount a distinct pair from its
own Secret (api.encryptionKeys):
- New install: use a distinct pair (
api.encryptionKeys.enabled=true) and keep it in every later upgrade. - Existing install: nothing to do.
api.encryptionKeys.enabledisfalseby default, so the upgrade keeps the current pair. Do not switch it on without a change of pair.
Change the encryption pair
To move to a distinct pair, or to replace a pair that may have leaked, keep the old private key mounted while the API re-encrypts the stored secrets at startup:
- Create the new pair and its Secret
kcm-api-encryption-keys. - Upgrade with the new pair, and the old private key as the previous key:
helm upgrade kcm oci://registry-1.docker.io/kafkaetech/kcm --version 1.5.1 -n kcm --reuse-values \ --set api.encryptionKeys.enabled=true \ --set api.env.KCM_ENCRYPT_PREVIOUS_PRIVATE_KEY=/app/keys/private.pem - Check the API log:
Key change: N secret(s) re-encrypted with the current key, and no line saying that secrets are encrypted with another key. - At the next upgrade, clear the previous key:
--set api.env.KCM_ENCRYPT_PREVIOUS_PRIVATE_KEY="".
If the old pair may have been read by someone else, also change the secrets themselves at their source (Kafka SASL passwords, Schema Registry, Kafka Connect and ksqlDB credentials, SSO client secrets, Git and Confluent Cloud credentials).
Version notes
Only the versions that ask for an action, or change a behaviour you may rely on, are listed. Every other upgrade needs no configuration change.
To chart 1.5.1
- The Ingress of the Helm chart now routes the SSO paths to the API (
/oauth2/,/login/oauth2/,/login/saml2/,/saml2/,/logout/saml2/). Upgrade the chart to 1.5.1 or later to use SSO behind its Ingress. With your own Ingress, add these paths yourself, see SSO.
To 1.5.0
- No configuration change is needed from 1.4.2.
- New editions: Community, Starter, Team and Enterprise. If your license allows fewer resources than you configured, KCM keeps everything running but blocks additions and changes until the extra ones are removed; a banner says what to remove. See Editions and licensing.
- Service accounts and their tokens need the Team edition or above.
- A new password is asked at the first sign in of a user created by an administrator, after an
administrator sets a password, and for the built-in
adminwhen its password was generated. - Kafka Connect clusters, Schema Registries and ksqlDB endpoints must have unique names within their Kafka
cluster: at the first start, duplicates are renamed
name-2,name-3... (logged and audited). - The consumer page never commits consumer offsets, and applies masking before any filter.
To 1.4.x
- 1.4.0, access control: a role on a KCM group now applies to its clusters, and a role on a cluster to its Kafka Connect clusters, ksqlDB endpoints and Schema Registries. Topic, consumer group, connector and subject grants now add access. A cluster VIEWER or EDITOR who also had such grants now sees every resource of the cluster: the API writes the list of users concerned in its log at startup. Review it after the upgrade.
- 1.4.0: a user without access to a cluster now gets a forbidden error instead of not found.
- 1.4.0: service account tokens are now opaque and revocable. Tokens issued before stay valid
until they expire while
KCM_SA_LEGACY_JWT_ACCEPTEDistrue. - 1.4.1: new optional variable
KCM_AUDIT_LOG_ENABLED(Enterprise).
To 1.3.x
- 1.3.0: schemas created from KCM keep the type chosen in the form. Before, they were all registered as Avro: check the subjects you created from KCM with an older version.
To 1.2.x
- 1.2.0: Schema Registry and self-managed Kafka Connect connections now verify TLS certificates and host names. If yours use a self-signed or private CA certificate, add the CA in Settings > Trusted certificates (no restart needed).
- 1.2.1: security release, upgrading is strongly recommended. Deploy kcm-api and kcm-ui 1.2.1 or later together. Kafka users and ACLs, Schema Registry connections and client quotas need the ADMIN role on the cluster. Changing the host of a cluster, Schema Registry, Kafka Connect or Kubernetes connection asks for its password or token again.
- 1.2.1, Docker Compose: the API generates its own key pairs at the first start. If you installed 1.2.0 or earlier from the Compose repository, follow Regenerate the Compose keys below.
- 1.2.1, Helm: if your key Secret was created from the files of the Compose repository, replace it with your own pair and follow Change the encryption pair.
Regenerate the Compose keys (from 1.2.0 or earlier)
Up to 1.2.0 the Compose repository shipped a sample key pair. Consider it compromised: from 1.2.1 the API refuses to start with it as its session signing pair.
- Update the repository (
git pull) and leave your olddata/keys/private.pemin place for now. docker compose pull && docker compose up -d. The API generates two new pairs indata/keys/jwtanddata/keys/encryption, and re-encrypts every stored secret with the new encryption pair. Every user signs in again.- Check the API log (
docker logs kcm-api): no line saying that secrets are encrypted with another key. - Delete the old
data/keys/private.pemanddata/keys/public.pem. - Change the secrets stored in KCM at their source, then enter them again in KCM.
- Change the
adminpassword if it is still the default one.
To 1.1.x
- 1.1.0: kcm-api and kcm-ui share one version number.
- 1.1.2: the public Helm chart moves to
oci://registry-1.docker.io/kafkaetech/kcm, with the chart version equal to the KCM version. The chart has no default passwords any more: on upgrade, the passwords of the deployed Secrets are kept. Change theadminpassword if it is stilladmin.
Within 1.0.x
- 1.0.21: secrets are encrypted at rest, existing values once at the first start. From this version on, back up the encryption key pair with the database.
- 1.0.23: optional distinct encryption pair in the Helm chart (
api.encryptionKeys).