The Kafka Producer allows you to publish messages in near-real-time across worker nodes where multiple, subscribed members have access. A Kafka Producer step publishes a stream of records to one Kafka topic.
Enter the following information in the transformation step name field.
- Step name: Specifies the unique name of the transformation on the canvas. The Step Name is set to Kafka Producer by default.
The Kafka Producer step features a Kafka connection setup tab and a configuration property options tab. Each tab is described below.
Fill in the following fields.
Select a connection type:
|Client ID||The unique Client identifier, used to identify and set up a durable connection path to the server to make requests and to distinguish between different clients.|
|Topic||The category to which records are published.|
|Key Field||In Kafka, all messages can be keyed, allowing for messages to be distributed to partitions based on their keys in a default routing scheme. If no key is present, messages are randomly distributed to partitions.|
|Message Field||The individual record contained in a topic.|
Use this tab to configure the Kafka Producer broker sources. For further information on these input names, see the Apache Kafka documentation site: https://kafka.apache.org/documentation/.
Metadata injection support
All fields of this step support metadata injection. You can use this step with ETL metadata injection to pass metadata to your transformation at runtime.