Packaging the Pentaho Worker Nodes Product

Add Custom Configuration Files and Plugins

Once you have planned your Worker Nodes setup, use the following guidelines to add custom configuration files and plugins.

It is recommended that you thoroughly check your current configuration. If you modify your configuration, you will need to repackage the Pentaho Worker Nodes Product.

Within the pentaho-ee-foundry/jobs/pdi-job-plugin/docker/directory add your custom configuration files and plugins in the following locations:

• OSGi  Kettle and Platform Plugins: Drop any OSGi kettle plugins in the data-integration/system/karaf/deploy directory.
• Customized Configuration Files: Place customized configuration files in the same location on the directory structure as they are in the main data-integration directory structure. For example:
  • The log4j.xml file should go under data-integration/classes, just like in a default build of Pentaho Data Integration (PDI).
  • The kettle.properties and repositories.xml files should go under .kettle, just like in a default build of PDI.
• Legacy Kettle Plugins and Kettle Step Plugins: Drop any legacy kettle plugins and kettle step plugins in the directory structure in the data-integration/plugins  directory.

Set a Repository Definition

To avoid rebuilding and redeploying images when different IP and ports are required, set the following explicit repository definition in your repositories.xml file as follows:

<repository>
   <id>PentahoEnterpriseRepository</id>
   <name>pentaho-repo</name>
   <description>Pentaho repository</description>
   <is_default>true</is_default>
   <repository_location_url>http://REPO_IP:REPO_PORT/pentaho</repository_location_url>
   <version_comment_mandatory>N</version_comment_mandatory>
</repository>

 In the above code definition, "pentaho-repo", "REPO_IP" and "REPO_PORT" are keys managed by the Pentaho Worker Nodes. Be sure to set these keys as-is.

Licenses are a mandatory group of files that must be placed in the folder structure to properly create the Worker Nodes. See Download the Required Licenses for more information. 

Download the Required Licenses

You will need to download the Pentaho Enterprise licenses used by your business and the Pentaho for HCI trial license.

• Download and copy the pentaho-ee licenses into the /pentaho-ee-foundry/jobs/pdi-job-plugin/docker/licenses folder.

For more information about how to download Pentaho licenses using the command line, see Manage Licenses using the Command Line Interface.

• Navigate to the location where you unpacked the pentaho-workernodes-8.1.0.0.0-XXX.tar.gz file. Copy the pentaho-hci-trial-license.plk file into the /pentaho-ee-foundry/services/worker-node-service/docker/licenses folder.

We recommend you use the pentaho-hci-trial-license.plk license when deploying the product.  This license is only valid for using the Pentaho functionality within HCI and is valid for a period of one year.

Build Worker Nodes Service Components

At this point, apply your configuration changes as discussed in Customize Worker Nodes.

1. Navigate to the pentaho-ee-foundry folder and run the ./create-pentaho-product-components.sh script.

If your site is behind a proxy firewall, you may need to adjust your docker file and component shell scripts to allow for passing a proxy value. If this situation applies to you, see Troubleshooting: Adjust the Docker File and Component Shell Scripts to Pass Proxy Values.

This script performs the following:

• Creates the Docker images for Worker Nodes job and saves the image tar.gz file to the  pentaho-ee-foundry/jobs/pdi-job-plugin/service-components folder.
• Creates the Docker images for Content Execution Broker services and saves the image tar.gz file to the  pentaho-ee-foundry/services/content-execution-broker/service-components folder.
• Under the service-components folder, creates the service plugin archive using the following three components: 
  • docker image
  • service plugin jar 
  • pkg tar.gz 
• Under the job-components folder, creates the job plugin archive using the following three components:
  • docker image
  • job plugin jar
  • pkg tar.gz

2. Check that the service plugin archive and the job plugin archive are copied to the pentaho-ee-foundry/product-components directory.

Build Required Keys to Build the Worker Nodes Product

When you start HCI, you must specify a public/private key used to sign in and verify the update package. While there are multiple ways to generate the required keys, we suggest you use the GNU Privacy Guard (GPG) tool with the following steps.

1. Using the tool GPG tool (also known as GnuPG), generate the keys. Follow the instruction and note the USER_NAME and password. 

gpg2 --gen-key

2. Export the public key using the USER_NAME chosen during the key generation. This export will pipe the value into a file called public.key.

gpg2 --export -a "<USER_NAME>" > public.key

3. Export the private key using the USER_NAME chosen during the key generation. This export will pipe the value into a file called private.key.

gpg2 --export-secret-keys -a "<USER_NAME>" > private.key

4. Optionally, if the GPG2 --gen-key fails, follow these additional steps:

1. Run the following command: 

sudo apt install  pinentry-qt

2. Unset DISPLAY.
3. Even though GPG2 instructs moving the mouse, typing commands, or doing something else with the system, the key generation steps can take a long time. To reduce this time, you can install the random number generation tool using the following command: 

sudo apt install rng-tools

5. Copy the newly created public.key and private.key files to the the  /pentao-ee-foundry directory.

Add the HCI Assembly Artifact and Package-Tool Files

In this task, you will add the HCI and Package-Tool files to your Pentaho worker nodes build directories.

This task assumes you have acquired the HCI Assembly Artifact and Package-Tool files from your Pentaho Enterprise Sales representative.  

1. Copy the the hci-1.3.0.93.tgz and package-tool-1.3.0.93.tgz files into the pentaho-ee-foundry/ directory prior to packaging.
2. Create a directory called 'package-tool' under the pentaho-ee-foundry/ directory.
3. Unpack the Package-tool archive into the newly created package-tool directory.

Build the Pentaho Worker Nodes File

You will now build the PentahoWorkerNodes product by executing the script  ./create-pentaho-product.sh. This script expects the public.key to be in the correct folder. 

1. Check that the public.key is in the current /pentao-ee-foundry directory.
2. From the same directory, run the script create-pentaho-product.sh.

The execution of this script will build the Pentaho Worker Nodes Product.

Install on a Single Instance of HCI

After packaging the Pentaho Worker Nodes Product, see Install Pentaho Worker Nodes on a Single Instance of HCI for an example of how to install your package on a single instance of HCI.