Minimum ACL Permissions

The Ultimate Engineering Toolkit for Apache Kafka®
Search…
Configuration to run kPow secured with Kafka ACLs
Kafka has the ability to restrict access to objects and operations within a cluster through the use of Kafka Access Control Lists (ACLs). This is different to kPow's own Role Based Access Controls.
You can skip this page if you do not have Kafka ACLs enabled in your cluster/s
When your cluster is secured with Kafka ACLs the Kafka user identified by the cluster connection credentials provided to kPow will need to have a minimum level of permissions for kPow to operate.
kPow Required Permissions
kPow needs the ability to read and write internal topics and read internal groups.
kPow checks for the following internal topics in your primary cluster (the first bootstrap in your configuration) on startup and will attempt to create them if required:
1
__oprtr_metric_pt1m
2
__oprtr_snapshot_state
3
__oprtr_audit_log
4
oprtr.compute.metrics.v2-oprtr_metric_v2_pt1m-changelog
5
oprtr.compute.snapshots.v2-oprtr_snaphot_state_v2-changelog
6
oprtr.compute.snapshots.v2-oprtr_snapshot_materialized_v2-repartition
Copied!
You can manually create these topics if you prefer, see Create kPow Topics.
Once started, kPow creates two internal streaming compute applications:
1
oprtr.compute.metrics.v2
2
oprtr.compute.snapshots.v2
Copied!
At a minimum, kPow must be able to read and write to and from these internal topics, must be able to read as those groups, and must have permissions to describe clusters, topics, configuration, and groups.
A basic set of Kafka ACLs that allows kPow to operate provides ALLOW on the following:
Kafka Resource
Kafka ACL
Detail
Cluster
Describe
*
Cluster
DescribeConfigs
*
Cluster
Create
* (if not manually creating kpow topics)
Topic
Describe
*
Topic
DescribeConfigs
*
Topic
Read
* or kpow topics only
Topic
Write
* or kpow topics only
Group
Describe
*
Group
Read
* or kpow groups only
kPow does not read from or write to topics other than internal ones as a part of normal operation.
Feature Specific ACLS
The following ACLS are optional and only required if you intend to permit the associated kPow action.
See User Authorization for a description of kPow User Actions and Controls.
Kafka Resource
Kafka ACL
Required for User Action
Cluster
Alter
ACL_EDIT
Cluster
AlterConfigs
BROKER_EDIT
Cluster
Create
TOPIC_CREATE
Topic
AlterConfigs
TOPIC_EDIT
Topic
Create
TOPIC_CREATE
Topic
Delete
TOPIC_DELETE
Topic
Read
TOPIC_INSPECT
Topic
Write
TOPIC_PRODUCE
Group
Read / Delete
GROUP_EDIT
Configuring Kafka ACLS
Creating ACLs on a cluster with no existing ACL configuration can cause issues. 
Consult your cluster provider documentation first.
For example the Amazon MSK ACL Guide describes extra ACLs required to allow inter-broker replication, and suggests not to set CLUSTER level ACLs.
Create a file containing client configuration for a user who has permissions to create ACLs.
1
security.protocol=SASL_PLAINTEXT
2
sasl.mechanism=PLAIN
3
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
Copied!
The following commands use the kafka-acls.sh script provided by Apache Kafka to create the basic set of ACLs described above that allows kPow to operate plus the ALTER CLUSTER ACL that allows kPow to create and delete ACLs.
1
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Describe --cluster '*'
2
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation DescribeConfigs  --cluster '*'
3
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Create --cluster '*'
4
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Write --topic '*'
5
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Read --topic '*'
6
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Describe --topic '*'
7
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation DescribeConfigs --topic '*'
8
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Describe --group '*'
9
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Read --group '*'
10
./kafka-acls.sh --bootstrap-server 127.0.0.1:9092 --command-config client.conf --add --allow-principal User:kpow --operation Alter --cluster '*'
Copied!
That set of ACLs can then be listed using kafka-acls.sh.
1
./kafka-acls.sh -bootstrap-server 127.0.0.1:9092 --command-config client.conf --list
2
​
3
Current ACLs for resource `ResourcePattern(resourceType=GROUP, name=*, patternType=LITERAL)`:
4
 	(principal=User:kpow, host=*, operation=READ, permissionType=ALLOW)
5
	(principal=User:kpow, host=*, operation=DESCRIBE, permissionType=ALLOW)
6
​
7
Current ACLs for resource `ResourcePattern(resourceType=TOPIC, name=*, patternType=LITERAL)`:
8
 	(principal=User:kpow, host=*, operation=READ, permissionType=ALLOW)
9
	(principal=User:kpow, host=*, operation=DESCRIBE, permissionType=ALLOW)
10
	(principal=User:kpow, host=*, operation=WRITE, permissionType=ALLOW)
11
	(principal=User:kpow, host=*, operation=DESCRIBE_CONFIGS, permissionType=ALLOW)
12
​
13
Current ACLs for resource `ResourcePattern(resourceType=CLUSTER, name=kafka-cluster, patternType=LITERAL)`:
14
 	(principal=User:kpow, host=*, operation=DESCRIBE_CONFIGS, permissionType=ALLOW)
15
	(principal=User:kpow, host=*, operation=DESCRIBE, permissionType=ALLOW)
16
	(principal=User:kpow, host=*, operation=CREATE, permissionType=ALLOW)
17
	(principal=User:kpow, host=*, operation=ALTER, permissionType=ALLOW)
Copied!
Create kPow Topics
Using the same client configuration file as above, the following script correctly creates the required internal kPow topics. You can run this script and create these topics on the primary cluster in which you expect kPow to keep data.
1
./kafka-topics.sh --create \
2
         --bootstrap-server 127.0.0.1:9092 \
3
         --command-config client.conf \
4
         --topic __oprtr_audit_log \
5
         --config compression.type=gzip \
6
         --config segment.bytes=104857600 \
7
         --config retention.ms=-1
8
./kafka-topics.sh --create \
9
         --bootstrap-server 127.0.0.1:9092 \
10
         --command-config client.conf \
11
         --topic __oprtr_metric_pt1m \
12
         --config compression.type=gzip \
13
         --config segment.bytes=104857600 \
14
         --config retention.ms=86400000 \
15
         --config segment.ms=43200000
16
./kafka-topics.sh --create \
17
         --bootstrap-server 127.0.0.1:9092 \
18
         --command-config client.conf \
19
         --topic __oprtr_snapshot_state \
20
         --config compression.type=gzip \
21
         --config segment.bytes=104857600 \
22
         --config retention.ms=86400000 \
23
         --config message.timestamp.type=LogAppendTime \
24
         --config segment.ms=43200000
25
./kafka-topics.sh --create \
26
         --bootstrap-server 127.0.0.1:9092 \
27
         --command-config client.conf \
28
         --topic oprtr.compute.metrics.v2-oprtr_metric_v2_pt1m-changelog \
29
         --config cleanup.policy=compact,delete \
30
         --config segment.bytes=52428800 \
31
         --config retention.ms=5400000 \
32
         --config message.timestamp.type=CreateTime \
33
         --config segment.ms=1800000
34
./kafka-topics.sh --create \
35
         --bootstrap-server 127.0.0.1:9092 \
36
         --command-config client.conf \
37
         --topic oprtr.compute.snapshots.v2-oprtr_snaphot_state_v2-changelog \
38
         --config cleanup.policy=compact,delete \
39
         --config segment.bytes=26214400 \
40
         --config retention.ms=604800000 \
41
         --config message.timestamp.type=CreateTime \
42
         --config segment.ms=604800000
43
./kafka-topics.sh --create \
44
         --bootstrap-server 127.0.0.1:9092 \
45
         --command-config client.conf \
46
         --topic oprtr.compute.snapshots.v2-oprtr_snapshot_materialized_v2-repartition \
47
         --config cleanup.policy=delete \
48
         --config segment.bytes=52428800 \
49
         --config retention.ms=-1 \
50
         --config message.timestamp.type=CreateTime
Copied!
Installation - Previous
Deployment Notes
Next - Installation
Application Logs
Last modified 4mo ago
Copy link
Contents
kPow Required Permissions
Feature Specific ACLS
Configuring Kafka ACLS
Create kPow Topics