Kafka Cluster

The Ultimate Engineering Toolkit for Apache Kafka®
Search…
Kafka Cluster
Connect kPow to Apache Kafka®
The ENVIRONMENT_NAME variable provides a UI friendly name for your Kafka cluster.
Requirements
kPow requires at least one configured Kafka cluster to operate.
When configuring Multi-Cluster installations the first cluster configured is your Primary Cluster and contains all the snapshot, metrics, and audit metadata for the installation.
A Kafka cluster can have multiple associated Schema registries and/or Kafka connect clusters.
Compatibility
kPow is compatible with Apache Kafka 1.0+.
kPow has been tested and is compatible with Apache Kafka, Amazon MSK, Red Had AMQ Streams, Red Hat OpenShift Streams for Apache Kafka, Aiven Managed Kafka, Instaclustr Managed Kafka, Confluent Platform, and Confluent Cloud.
kPow has been confirmed to work with Azure Event Hubs* and Redpanda, though these platforms are not officially supported and all kPow features may not be available.
*Some disk related metrics and telemetry are not available when using kPow Azure Event Hubs.
FIPS
kPow is capable of integrating with FIPS compliant Kafka clusters.
Contact [email protected] for assistance.
Access Control
User permissions to Kafka cluster resources are defined by Cluster actions.​
AWS IAM Integration
kPow supports IAM Access Control for AWS MSK.
Simply set your kPow connection fields appropriately, e.g.
1
SSL_TRUSTSTORE_LOCATION=<PATH_TO_TRUST_STORE_FILE>
2
SECURITY_PROTOCOL=SASL_SSL
3
SASL_MECHANISM=AWS_MSK_IAM
4
SASL_JAAS_CONFIG=software.amazon.msk.auth.iam.IAMLoginModule required;
5
SASL_CLIENT_CALLBACK_HANDLER_CLASS=software.amazon.msk.auth.iam.IAMClientCallbackHandler
Copied!
See the AWS documentation for more information, including JAAS config for named profiles.
Configuration
kPow connects to a Kafka with the same configuration as a Kafka consumer or producer.
This configuration may be familiar to you, and is provided to kPow by environment variables.
The list of connection variables follows, many are optional. See the Kafka client docs for more.
Need to create a Keystore from certificate files? This Stackoverflow answer might help.
Variable
Description
ENVIRONMENT_NAME
Optional, UI friendly label for this cluster and resources
CLUSTER_ID
Optional, unique identifier for the cluster. Required when connecting to Azure Event Hubs or Redpanda.
BOOTSTRAP
The Kafka cluster bootstrap URL
KAFKA_VARIANT
'EVENT_HUBS' for Azure Event Hubs or 'RHOSAK' for Red Had OpenShift Streams for Apache Kafka, otherwise do not set.
SECURITY_PROTOCOL
PLAINTEXT, SSL, SASL_PLAINTEXT, or SASL_SSL
SASL_MECHANISM
GSSAPI, AUTHBEARER, SCRAM, PLAIN,
SASL_JAAS_CONFIG
Java Authentication and Authorization Service config
SSL_KEYSTORE_LOCATION
The path to a keystore for auth with certificates
SSL_KEYSTORE_PASSWORD
The password to access the auth keystore
SSL_KEY_PASSWORD
The password of the key within the keystore
SSL_KEYSTORE_TYPE
The file format of the keystore file
SSL_KEYMANAGER_ALGORITHM
The key manager algorithm used for SSL 
SSL_TRUSTSTORE_LOCATION
The path to a truststore for auth with certificates
SSL_TRUSTSTORE_PASSWORD
The password to access the auth truststore
SSL_TRUSTSTORE_TYPE
The file format of the truststore file
SSL_TRUSTMANAGER_ALGORITHM
The trust manager algorithm user for SSL
SSL_ENDPOINT_IDENTIFICATION_ALGORITHM
Often required when authenticating via SSL
SSL_PROVIDER
Name of the security provider used for SSL
SSL_CIPHER_SUITES
A list of cipher suites
SSL_PROTOCOL
TLS, TLSv1.1, or TLSv1.2
SSL_ENABLED_PROTOCOLS
The list of protocols enabled for SSL
SSL_KEYSTORE_KEY
​
Private key in the format specified by SSL_KEYSTORE_TYPE. See: KIP-651​
SSL_KEYSTORE_CERTIFICATE_CHAIN
Certificate chain in the format specified by SSL_KEYSTORE_TYPE. See: KIP-651​
SSL_TRUSTSTORE_CERTIFICATES
Trusted certificates in the format specified by SSL_KEYSTORE_TYPE. See: KIP-651​
Configuration - Previous
Environment Variables
Next - Configuration
Schema Registry
Last modified 2mo ago
Copy link
Contents