Configuration

Kafka cluster

The ENVIRONMENT_NAME variable provides a UI friendly name for your Kafka cluster.

Requirements

Kpow requires at least one configured Kafka cluster to operate.

When configuring Multi-cluster management installations the first cluster configured is your Primary Cluster and contains all the snapshot, metrics, and audit metadata for the installation.

A Kafka cluster can have multiple associated Schema Registries and/or Kafka Connect clusters.

Compatibility

Kpow is compatible with Apache Kafka 1.0+.

Kpow has been tested and is compatible with Apache Kafka, Amazon MSK, Red Had AMQ Streams, Red Hat OpenShift Streams for Apache Kafka, Aiven Managed Kafka, Instaclustr Managed Kafka, Confluent Platform, Confluent Cloud, and Redpanda.

Kpow is supported for use with Redpanda where Redpanda meets the standard Kafka API and featureset.

FIPS

Kpow is capable of integrating with FIPS compliant Kafka clusters.

Contact [email protected] for assistance.

Access Control

User permissions to Kafka cluster resources are defined by Cluster actions. See: User authorization.

Configuration

Kpow connects to a Kafka with the same configuration as a Kafka consumer or producer.

This configuration may be familiar to you, and is provided to Kpow by environment variables.

The list of connection variables follows, many are optional. See the Kafka client docs for more.

Need to create a Keystore from certificate files? This Stackoverflow answer might help.

VariableRequiredDescription
BOOTSTRAPtrueThe Kafka cluster bootstrap URL
ENVIRONMENT_NAMEfalseOptional, UI friendly label for this cluster and resources
CLUSTER_IDfalseOptional, unique identifier for the cluster. Required when connecting to Azure Event Hubs or Redpanda.
KAFKA_VARIANTfalse'EVENT_HUBS' for Azure Event Hubs or 'RHOSAK' for Red Had OpenShift Streams for Apache Kafka, otherwise do not set.
SECURITY_PROTOCOLfalseSpecify the connection security protocol, acceptable values are: PLAINTEXT, SASL_PLAINTEXT, SASL_SSL, or SSL
SASL_MECHANISMfalseSASL mechanism used for client connections. This may be any mechanism for which a security provider is available. GSSAPI is the default mechanism.
SASL_JAAS_CONFIGfalseJAAS login context parameters for SASL connections in the format used by JAAS configuration files. JAAS configuration file format is described here. The format for the value is: 'loginModuleClass controlFlag (optionName=optionValue)*;'. For brokers, the config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.jaas.config=com.example.ScramLoginModule required;
SASL_CLIENT_CALLBACK_HANDLER_CLASSfalseThe fully qualified name of a SASL client callback handler class that implements the AuthenticateCallbackHandler interface.
SASL_KERBEROS_SERVICE_NAMEfalseThe Kerberos principal name that Kafka runs as. This can be defined either in Kafka's JAAS config or in Kafka's config.
SASL_LOGIN_CALLBACK_HANDLER_CLASSfalseThe fully qualified name of a SASL login callback handler class that implements the AuthenticateCallbackHandler interface. For brokers, login callback handler config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.callback.handler.class=com.example.CustomScramLoginCallbackHandler
SASL_LOGIN_CLASSfalseThe fully qualified name of a class that implements the Login interface. For brokers, login config must be prefixed with listener prefix and SASL mechanism name in lower-case. For example, listener.name.sasl_ssl.scram-sha-256.sasl.login.class=com.example.CustomScramLogin
SASL_OAUTHBEARER_JWKS_ENDPOINT_URLfalse-
SASL_OAUTHBEARER_TOKEN_ENDPOINT_URLfalse-
SASL_KERBEROS_KINIT_CMDfalseKerberos kinit command path.
SASL_KERBEROS_MIN_TIME_BEFORE_RELOGINfalseLogin thread sleep time between refresh attempts.
SASL_KERBEROS_TICKET_RENEW_JITTERfalsePercentage of random jitter added to the renewal time.
SASL_KERBEROS_TICKET_RENEW_WINDOW_FACTORfalseLogin thread will sleep until the specified window factor of time from last refresh to ticket's expiry has been reached, at which time it will try to renew the ticket.
SASL_LOGIN_CONNECT_TIMEOUT_MSfalse-
SASL_LOGIN_READ_TIMEOUT_MSfalse-
SASL_LOGIN_REFRESH_BUFFER_SECONDSfalseThe amount of buffer time before credential expiration to maintain when refreshing a credential, in seconds. If a refresh would otherwise occur closer to expiration than the number of buffer seconds then the refresh will be moved up to maintain as much of the buffer time as possible. Legal values are between 0 and 3600 (1 hour); a default value of 300 (5 minutes) is used if no value is specified. This value and sasl.login.refresh.min.period.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.
SASL_LOGIN_REFRESH_MIN_PERIOD_SECONDSfalseThe desired minimum time for the login refresh thread to wait before refreshing a credential, in seconds. Legal values are between 0 and 900 (15 minutes); a default value of 60 (1 minute) is used if no value is specified. This value and sasl.login.refresh.buffer.seconds are both ignored if their sum exceeds the remaining lifetime of a credential. Currently applies only to OAUTHBEARER.
SASL_LOGIN_REFRESH_WINDOW_FACTORfalseLogin refresh thread will sleep until the specified window factor relative to the credential's lifetime has been reached, at which time it will try to refresh the credential. Legal values are between 0.5 (50%) and 1.0 (100%) inclusive; a default value of 0.8 (80%) is used if no value is specified. Currently applies only to OAUTHBEARER.
SASL_LOGIN_REFRESH_WINDOW_JITTERfalseThe maximum amount of random jitter relative to the credential's lifetime that is added to the login refresh thread's sleep time. Legal values are between 0 and 0.25 (25%) inclusive; a default value of 0.05 (5%) is used if no value is specified. Currently applies only to OAUTHBEARER.
SASL_LOGIN_RETRY_BACKOFF_MAX_MSfalse-
SASL_LOGIN_RETRY_BACKOFF_MSfalse-
SASL_OAUTHBEARER_CLOCK_SKEW_SECONDSfalse-
SASL_OAUTHBEARER_EXPECTED_AUDIENCEfalse-
SASL_OAUTHBEARER_EXPECTED_ISSUERfalse-
SASL_OAUTHBEARER_JWKS_ENDPOINT_REFRESH_MSfalse-
SASL_OAUTHBEARER_JWKS_ENDPOINT_RETRY_BACKOFF_MAX_MSfalse-
SASL_OAUTHBEARER_JWKS_ENDPOINT_RETRY_BACKOFF_MSfalse-
SASL_OAUTHBEARER_SCOPE_CLAIM_NAMEfalse-
SASL_OAUTHBEARER_SUB_CLAIM_NAMEfalse-
SSL_KEY_PASSWORDfalseThe password of the private key in the key store file. This is optional for client.
SSL_KEYSTORE_CERTIFICATE_CHAINfalse-
SSL_KEYSTORE_KEYfalse-
SSL_KEYSTORE_LOCATIONfalseThe location of the key store file. This is optional for client and can be used for two-way authentication for client.
SSL_KEYSTORE_PASSWORDfalseThe store password for the key store file. This is optional for client and only needed if ssl.keystore.location is configured.
SSL_TRUSTSTORE_CERTIFICATESfalse-
SSL_TRUSTSTORE_LOCATIONfalseThe location of the trust store file.
SSL_TRUSTSTORE_PASSWORDfalseThe password for the trust store file. If a password is not set access to the truststore is still available, but integrity checking is disabled.
SSL_ENABLED_PROTOCOLSfalseThe list of protocols enabled for SSL connections.
SSL_KEYSTORE_TYPEfalseThe file format of the key store file. This is optional for client.
SSL_PROTOCOLfalseThe SSL protocol used to generate the SSLContext. Default setting is TLSv1.2, which is fine for most cases. Allowed values in recent JVMs are TLSv1.2 and TLSv1.3. TLS, TLSv1.1, SSL, SSLv2 and SSLv3 may be supported in older JVMs, but their usage is discouraged due to known security vulnerabilities.
SSL_PROVIDERfalseThe name of the security provider used for SSL connections. Default value is the default security provider of the JVM.
SSL_TRUSTSTORE_TYPEfalseThe file format of the trust store file.
SSL_CIPHER_SUITESfalseA list of cipher suites. This is a named combination of authentication, encryption, MAC and key exchange algorithm used to negotiate the security settings for a network connection using TLS or SSL network protocol. By default all the available cipher suites are supported.
SSL_ENDPOINT_IDENTIFICATION_ALGORITHMfalseThe endpoint identification algorithm to validate server hostname using server certificate.
SSL_ENGINE_FACTORY_CLASSfalse-
SSL_KEYMANAGER_ALGORITHMfalseThe algorithm used by key manager factory for SSL connections. Default value is the key manager factory algorithm configured for the Java Virtual Machine.
SSL_SECURE_RANDOM_IMPLEMENTATIONfalseThe SecureRandom PRNG implementation to use for SSL cryptography operations.
SSL_TRUSTMANAGER_ALGORITHMfalseThe algorithm used by trust manager factory for SSL connections. Default value is the trust manager factory algorithm configured for the Java Virtual Machine.