Data produce

Introduction

Kpow's data produce UI allows you to bulk-produce multiple records to a single topic.

The data produce form is idempotent. Once a record has been produced it won't be retried.

Metadata about the record appears in the footer once it has been produced.

If a record was unsuccessfully produced, information about the error will appear in the records footer:

Use the toolbar to filter down to just erroring records, or clear any produced records from the form:

Adding headers

Kpow supports multiple key/value pairs as headers to the record.

Tombstone records

Specify the 'None' serde for either the key or value serializer to produce a null key or value.

Specifying 'None' as the value serializer (with a non-null key) will produce a tombstone record.

Importing records

Load records from a CSV, JSON or EDN file into the data produce form by clicking the "Import Records" button on the control pane.

CSV

An example CSV:

"1000","{""name"": ""sam""}"
"1001","{""name"": ""jane""}"

Importing with headers

All remaining columns after the initial key and value column are reserved for headers:

"1000","{""name"": ""sam""}",HeaderKey1,HeaderValue1,HeaderKey2,HeaderValue2
"1001","{""name"": ""jane""}",HeaderKey1,HeaderValue2

JSON

The format for importing JSON is a collection of objects containing key and value keys. The values for the key/value pairs must be a string.

[{
	"key": "1000",
	"value": "{\"name\": \"sam\"}"
}]

Importing with headers

You can optionally specify a headers key for each record in the array. This must be a map where the keys are strings and the values are arrays of strings.

[{
	"key": "1000",
	"value": "{\"name\": \"sam\"}",
	"headers": {"header1": ["value1", "value2"]}
}]

Headers are structured like this because there can be multiple values for each key in a Kafka records header.

EDN

The format for importing EDN is a collection of maps containing :key and :value keys. The values for the key/value pairs must be a string

[{:key "1000", :value "{\"name\": \"sam\"}"}]

Importing with headers

You can optionally specify a :headers key for each record in the array. This must be a map where the keys are strings and the values are arrays of strings.

[{:key "1000",
  :value "{\"name\": \"sam\"}",
  :headers {"header1" ["value1"]}}]

Headers are structured like this because there can be multiple values for each key in a Kafka records header.

Data inspect

After executing a Data inspect query, you can click the "Produce Results" button on the control pane. This will load the results from the query into the data produce form.

You can also import a single record at a time by hovering over a data inspect result, clicking on the "Actions" dropdown and clicking "Produce".