Skip to main content
Version Pentaho
Hitachi Vantara Lumada and Pentaho Documentation

Pentaho Worker Nodes

  1. Last updated
  2. Save as PDF

Worker nodes allow you to execute Pentaho work items, such as PDI jobs and transformations, with parallel processing and dynamic scalability with load balancing and dependency management in a clustered environment. 

You can deploy worker nodes in physical, virtual, and Cloud environments to manage the base and peak time processing demands for the workloads of hundreds of work items (PDI jobs and transformations) at different times and with different processing requirements. Moreover, through the use of a hybrid deployment of environments, you can provide additional processing capabilities to distribute the load in cases when the on-premise resources are insufficient, by scaling out into the Cloud for more resources. 

Hitachi Content Intelligence (HCI) and other technologies, such as Docker and Mesos, provide the means to enable the Pentaho Server to scale in a worker node environment. You can manage system scaling by adding or removing instances (servers or virtual machines connected to the cluster) to the system and by specifying which services run on an instance. A single-instance system is useful for testing and demonstration purposes, while a multi-instance system is recommended for use in a production environment. 

distibute and scale.png

Audience

Worker Nodes documentation is intended for selected 8.1 Hitachi Vantara customers who are interested in exploring this feature. Contact your Hitachi Vantara sales associate for more information.

Worker node installation and configuration documentation is intended for IT administrators who know where data is stored, how to connect to it, details about the computing environment, and how to use the Linux command line. 

Requirements

You must meet the following requirements to set up Worker Nodes:

  • Pentaho 8.1 or later installation. See Pentaho Installation.
  • Pentaho Worker Nodes distribution file.
  • A 64-bit Linux distribution
  • Docker version 17.05.0-ce (Minimum version required) or later
  • Java JDK 8.0

Process Overview

The following topics explain how to set up HCI and Pentaho to use worker nodes:

  1. Read the Pentaho Worker Nodes System Recommendations. We can recommend several hardware, networking, and operating systems for creating and running your version of the Pentaho Worker Nodes Product on one or more instances.
  2. Packaging the Pentaho Worker Nodes Product. Package a version of the Pentaho Worker Nodes Product customized to your system.
  3. Install Pentaho Worker Nodes on a Single Instance of HCI. Walk through how to install your custom Pentaho Worker Nodes Product package on a single instance of HCI.
  4. Set Up Pentaho Worker Nodes on Pentaho Server. Enable and configure for your Pentaho Worker Node Product package on your instance of HCI along with establishing secure communications.

Once you have Pentaho Worker Nodes set up and configured, you can then Run Work Items on Pentaho Worker Nodes and Administer Pentaho Worker Nodes.