Kafka clusters

Group your clusters by environment, register them with their security settings, and check the connection before you save.

Environments

An environment (also called a KCM group) holds Kafka clusters that belong together, for example production, staging or a region. Each one has a name and a color, used in the header so you always know where you are. Roles granted on an environment apply to all its clusters.

  1. Open Environments and click Add.
  2. Give it a name, for example production, and a color.
  3. Open it to see its clusters.
KCM environments page listing environments with their colors
Environments group clusters and carry roles.

Creating an environment needs the global ADMIN role; editing it needs ADMIN on the environment.

Register a Kafka cluster

  1. In the environment, click Add.
  2. General: a name (unique in the environment) and the bootstrap servers. Optionally turn on metrics, set a consumer group prefix (KCM_ by default) for the temporary groups of KCM, and topics to exclude from metrics.
  3. Security: the protocol and its credentials (see below).
  4. Create, then Test connection.
Add Kafka cluster dialog with name, bootstrap servers and metrics
General settings of a cluster.
Security settings with protocol, SASL mechanism and credentials
Security settings: protocol, mechanism, credentials and truststore.

Creating, editing or deleting a cluster needs ADMIN on its environment. Changing the host of a cluster asks for its password again. Deleting a cluster removes only the truststore and keytab files that KCM stored itself.

Security protocols

ProtocolWhat to provide
PLAINTEXTNothing (test and local clusters only).
SSLThe truststore of the brokers and its password; a keystore for mutual TLS.
SASL_PLAINTEXT, SASL_SSLThe mechanism (PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, GSSAPI for Kerberos, and AWS_MSK_IAM with SASL_SSL) and its credentials or JAAS configuration; for SASL_SSL, the truststore.

Test the connection

Edit Kafka cluster dialog with the Test connection button
Test connection reports the error from the brokers, if any.

The test logs the full error in the API log. See Troubleshooting when it fails.

Trusted certificates

In Settings > Trusted certificates, global ADMINs add the CA certificates (PEM) of your private certificate authorities, or of a TLS inspecting proxy. They apply at once, without restart, to every HTTPS call of KCM: Schema Registry, Kafka Connect, ksqlDB, Confluent Cloud APIs and the AI assistant. Kafka brokers use the truststore of each cluster instead.