Skip to main content
Hitachi Vantara Lumada and Pentaho Documentation

Packaging the Pentaho Worker Nodes Product

Before you install Pentaho Worker Nodes and HCI, consider how you want your system to operate. Below is an example configuration of the Pentaho Worker Nodes product on a typical customer site. Your IT Administrator will need to establish access between the Pentaho Server, the Pentaho Repository, and all product instances you want to run (worker nodes plus services), regardless of where each node is located. For example, in the same way that the Pentaho Server needs to talk to the Pentaho Repository, the worker node container requires the same ability. All the custom configurations you would apply to the Pentaho Server apply to the worker node container you are building. 

So, it is mandatory to configure your repository, and all the databases you might be using or want to use, in the worker node environment. Since the repository will be accepting multiple remote connections from the worker nodes and the Pentaho Server, you must configure the repository database to accept multiple remote connections. See Customize Worker Nodes for further information.


Building the HCI Assembly Artifact

Before you can install and configure Worker Nodes, you must build the HCI Assembly Artifact.

The artifact must meet the following requirements:

  • Java JDK 1.8. Note: Later Java releases are not supported.
  • 64-bit Linux distribution
  • Docker version 17.05.0-ce  or later

The product containing the necessary scripts to package the worker-node service and to package the final product can be downloaded from the Customer Support Portal under the name:

Access to this ZIP file and the Pentaho Worker Nodes product is available to select customers. Contact your Hitachi Vantara sales associate for more information.  

Since the Pentaho Worker Nodes Product is a distribution package, follow the instructions below to unpack it.

  1. On the server where you want to deploy the Pentaho Worker Nodes Product, open a command line interface and navigate to the folder where the is located.  For example:

cd /home/user/Downloads 

  1. Extract the file. 


  1. Navigate to the workernodes folder that was created from the previous step.

cd  pentaho-workernodes-

  1. Run the EULA installer.

java -DINSTALL_PATH=pentaho-worker-nodes -DEULA_ACCEPT=true -jar pentaho-workernodes- -options-system 

  1. When complete, navigate to the INSTALL_PATH (pentaho-worker-nodes in the above example):

cd pentaho-worker-nodes

6.    Extract the worker nodes archive using the following command:

tar -zxf pentaho-workernodes-

  1. Download the file. 

File names and licenses are checked during the component build process. Missing or incorrectly named files will halt the build process. Be sure to check the accuracy of your files and licenses prior to downloading.

  1. Copy the  pdi-ee-client archive into the pentaho-ee-foundry/jobs/pdi-job-plugin/docker/directory.  For example:

cp ~/Downloads/ pentaho-ee-foundry/jobs/pdi-job-plugin/docker/

Customize Worker Nodes

Before running the build scripts, you must add any customizations for the Worker Nodes, which can range from simple configuration file changes to complex custom plugins. For example, a simple configuration file change might include a debugging file that is required to have a worker node communicate with the repository you are using, while a more complex customization might include a custom PDI plugin that you want the worker node to be able to use. 

Add Custom Configuration Files and Plugins

Once you have planned your Worker Nodes setup, use the following guidelines to add custom configuration files and plugins.

It is recommended that you thoroughly check your current configuration. If you modify your configuration, you will need to repackage the Pentaho Worker Nodes Product.

Within the pentaho-ee-foundry/jobs/pdi-job-plugin/docker/directory add your custom configuration files and plugins in the following locations:

  • OSGi  Kettle and Platform Plugins: Drop any OSGi kettle plugins in the data-integration/system/karaf/deploy directory.
  • Customized Configuration Files: Place customized configuration files in the same location on the directory structure as they are in the main data-integration directory structure. For example:
    • The log4j.xml file should go under data-integration/classes, just like in a default build of Pentaho Data Integration (PDI).
    • The and repositories.xml files should go under .kettle, just like in a default build of PDI.
  • Legacy Kettle Plugins and Kettle Step Plugins: Drop any legacy kettle plugins and kettle step plugins in the directory structure in the data-integration/plugins  directory.

Best Practices for Applying Customizations

It is recommended that you do not explicitly reference IP and port values in your XML files so you do not unintentionally bind your Docker images to these specific values. Otherwise, your Docker images would be unusable in different deployment environments that require different IPs and ports. For example, the IP and port you use to set up Worker Nodes for demonstration purposes differs from the IP and port you use in a production environment.

Set a Repository Definition

To avoid rebuilding and redeploying images when different IP and ports are required, set the following explicit repository definition in your repositories.xml file as follows:

   <description>Pentaho repository</description>

 In the above code definition, "pentaho-repo", "REPO_IP" and "REPO_PORT" are keys managed by the Pentaho Worker Nodes. Be sure to set these keys as-is.

Licenses are a mandatory group of files that must be placed in the folder structure to properly create the Worker Nodes. See Download the Required Licenses for more information. 

Building the Pentaho Worker Nodes Product File

To build the Pentaho Worker Nodes product, you will need both the HCI Assembly Artifact  and Package-Tool files, and the Pentaho license files. The instructions for downloading the license files are below. The HCI Assembly Artifact and Package-Tool files are the property of HCI. Contact your Pentaho Enterprise Sales representative to obtain these files.

There are several sections within these instructions, including:

Download the Required Licenses

You will need to download the Pentaho Enterprise licenses used by your business and the Pentaho for HCI trial license.

  • Download and copy the pentaho-ee licenses into the /pentaho-ee-foundry/jobs/pdi-job-plugin/docker/licenses folder.

For more information about how to download Pentaho licenses using the command line, see Manage Licenses using the Command Line Interface.

  • Navigate to the location where you unpacked the pentaho-workernodes- file. Copy the pentaho-hci-trial-license.plk file into the /pentaho-ee-foundry/services/worker-node-service/docker/licenses folder.

We recommend you use the pentaho-hci-trial-license.plk license when deploying the product.  This license is only valid for using the Pentaho functionality within HCI and is valid for a period of one year.

Build Worker Nodes Service Components

At this point, apply your configuration changes as discussed in Customize Worker Nodes.

  1. Navigate to the pentaho-ee-foundry folder and run the ./ script.

If your site is behind a proxy firewall, you may need to adjust your docker file and component shell scripts to allow for passing a proxy value. If this situation applies to you, see Troubleshooting: Adjust the Docker File and Component Shell Scripts to Pass Proxy Values.

This script performs the following:

  • Creates the Docker images for Worker Nodes job and saves the image tar.gz file to the  pentaho-ee-foundry/jobs/pdi-job-plugin/service-components folder.
  • Creates the Docker images for Content Execution Broker services and saves the image tar.gz file to the  pentaho-ee-foundry/services/content-execution-broker/service-components folder.
  • Under the service-components folder, creates the service plugin archive using the following three components: 
    • docker image
    • service plugin jar 
    • pkg tar.gz 
  • Under the job-components folder, creates the job plugin archive using the following three components:
    • docker image
    • job plugin jar
    • pkg tar.gz
  1. Check that the service plugin archive and the job plugin archive are copied to the pentaho-ee-foundry/product-components directory.

Build Required Keys to Build the Worker Nodes Product

When you start HCI, you must specify a public/private key used to sign in and verify the update package. While there are multiple ways to generate the required keys, we suggest you use the GNU Privacy Guard (GPG) tool with the following steps.

  1. Using the tool GPG tool (also known as GnuPG), generate the keys. Follow the instruction and note the USER_NAME and password. 

gpg2 --gen-key

  1. Export the public key using the USER_NAME chosen during the key generation. This export will pipe the value into a file called public.key.

gpg2 --export -a "<USER_NAME>" > public.key

  1. Export the private key using the USER_NAME chosen during the key generation. This export will pipe the value into a file called private.key.

gpg2 --export-secret-keys -a "<USER_NAME>" > private.key

  1. Optionally, if the GPG2 --gen-key fails, follow these additional steps:
  1. Run the following command: 

sudo apt install  pinentry-qt

  1. Unset DISPLAY.
  2. Even though GPG2 instructs moving the mouse, typing commands, or doing something else with the system, the key generation steps can take a long time. To reduce this time, you can install the random number generation tool using the following command: 

sudo apt install rng-tools

  1. Copy the newly created public.key and private.key files to the the  /pentao-ee-foundry directory.

Add the HCI Assembly Artifact and Package-Tool Files

In this task, you will add the HCI and Package-Tool files to your Pentaho worker nodes build directories.

This task assumes you have acquired the HCI Assembly Artifact and Package-Tool files from your Pentaho Enterprise Sales representative.  

  1. Copy the the hci- and package-tool- files into the pentaho-ee-foundry/ directory prior to packaging.
  2. Create a directory called 'package-tool' under the pentaho-ee-foundry/ directory.
  3. Unpack the Package-tool archive into the newly created package-tool directory.

Build the Pentaho Worker Nodes File

You will now build the PentahoWorkerNodes product by executing the script  ./ This script expects the public.key to be in the correct folder. 

  1. Check that the public.key is in the current /pentao-ee-foundry directory.
  2. From the same directory, run the script

The execution of this script will build the Pentaho Worker Nodes Product.

Install on a Single Instance of HCI

After packaging the Pentaho Worker Nodes Product, see Install Pentaho Worker Nodes on a Single Instance of HCI for an example of how to install your package on a single instance of HCI.