Skip to main content
Hitachi Vantara Lumada and Pentaho Documentation

Kafka Offset

You can use the Kafka Offset job entry to reset the offset value of all partitions in a topic based upon a given timestamp or epoch value.


The following field is general to this job entry:

  • Job Name: Specify the unique name of the Kafka Offset entry on the canvas. You can customize the name or leave it as the default.


The Kafka Offset entry includes three tabs for the step. Each tab is described below.

Setup tab

In this tab, define the connections used for receiving messages, topics to which you want to subscribe, and the consumer group for the topics.

Bootstrap servers

Specify the Bootstrap servers from which you want to receive the Kafka streaming data.

TopicsEnter the name of each Kafka topic from which you want to consume streaming data (messages). You must include all topics that you want to consume.
Consumer group

Enter the name of the group of which you want this consumer to be a member. Each Kafka Consumer step will start a single thread for consuming.

When part of a consumer group, each consumer is assigned a subset of the partitions from topics it has subscribed to, which locks those partitions. Each instance of a Kafka Consumer step will only run a single consumer thread.

Options tab

Use this tab to send the parameters to the broker when connecting. The parameter values can be encrypted. A few of the most common options are included for your convenience. You can enter any desired Kafka property. For further information on these input names, see the Apache Kafka documentation site:

Offset Settings tab

To reset the offset value, set the start date in the Offset timestamp field and select the format in the Timestamp format field.

Offset timestamp

Specifies the start date timestamp to use as the offset. The timestamp can be a past or future time. When this job is run, the offset value in each partition will be reset to timestamp given.

Timestamp format

Specifies the timestamps format for the offset.

When epoch value is given, the timestamp format is not needed.


The kafka_job_abort.kjb sample job in the design-tools/data-integration/plugins/kafka-job-plugins-zip- directory demonstrates the capabilities of this job using the Abort step.

The kafka_job.kjb sample job in the design-tools/data-integration/plugins/pentaho-streaming-kafka-plugin-zip- directory demonstrates in the kafka_job.kjb job with both successes and failures. Comments in the transformations explain how the steps are used.