Setting up mounted volumes on Pentaho Worker Nodes
You can use mounted volumes to manage your Pentaho Worker Nodes configuration and expanding default capabilities. Mounted volumes are a native Docker capability. For more information about mounted volumes, see the Docker documentation on Volumes.
Pentaho Worker Nodes leverage mounted volumes capability for the following tasks:
- Adding or updating expired Pentaho licenses.
- Adding a new PDI step plugin.
- Adding custom JAR files.
- Changing the details of a Pentaho Repository connection.
The files in the mounted volume have the following structure:
<some-dir> |--- .kettle ( <dot><kettle> ) |--- custom-lib |--- data |--- licenses |--- plugins |--- plugins-osgi
Creating the folder structure
You can create the folder structure by running a utility script located in the Pentaho Worker Nodes installation folder.
Procedure
In your Pentaho Worker Nodes installation directory, navigate to <product-install_dir>/jobs/com.pentaho.foundry.plugins.job.pdi/<pdi_job_version>/package/svcbin/util/create-kettle-volume-dir.sh.
Run the script: ./create-kettle-volume-dir.sh<path-to-some-dir> to create the folder structure under the <path-to-some-dir>.
Move the following files to the specifed folder in the folder structure:
File(s) Folder location kettle.properties, repositories.xml, jdbc.properties, and log4j.xml files .kettle Custom JAR files for PDI runtime custom-lib input files used in KTR and KJB execution data .installedLicenses.xml licenses PDI plugins plugins PDI OSGI plugins or features.cfg files plugins-osgi
Required content in mounted volume
There is a minimum set of required content that you must provide for Pentaho Worker Nodes to operate properly. Use the table below to place the specified files properly in the mounted volume directory.
File | Folder location | Description |
repositories.xml | .kettle | This file contains the details of the Pentaho Repository connection. It is required when connecting to the Pentaho Repository |
kettle.properties | .kettle | This file contains the Kettle engine configuration settings. |
installedLicenses.xml | .kettle/licenses | This file contains your valid Pentaho license(s). The PDI Enterprise Edition license is required. |
Setting up a mounted volume collection
You can declare your own mounted volumes so you can add content to use during execution, such as CSV files. You can also add KTR and KJB files for direct execution.
Create additional Docker volumes
Similar to creating the pdi_kettlevolume
, you can create
multiple Docker volumes that you can mount inside the container.
Procedure
Run the following command:
> docker volume create --driver local --opt type=<type-of-volume-?> --opt o=addr=<host-ip>,rw --opt device=:<some-dir>/.kettle <docker-volume-name>
Declare any volumes (besides the
The collection of volumes appears under /root/volumes/ inside the container.pdi_kettle
volume) that you would like mounted inside of the container as part of the PDI Service/Jobvolume collection
in Foundry.(Optional) Because of the way the volume is mounted, you may need to assign write permissions to folders for others (o+w) for any folders serving as the mount point for a volume, specifically in an NFS volume case.
The docker group is not guaranteed ownership of the folder inside of the container.NoteCurrently, PDI Service and PDI Job have different ways of declaring a collection of volumes because of the architectural differences between a Foundry service and Job.
Results
Example: The sample.csv file is located in your file system at /home/user/sample.csv. You create a Docker volume named csv-samples pointing to /home/user/. This volume is declared in Foundry for a PDI Service/Job.
When properly set up, you would see the sample.csv file at /root/volumes/csv-samples/sample.csv inside of the container.
Declare additional volumes for a PDI Service
Additional volumes can be declared under the Volumes tab of the PDI Service. Any volumes you add must be declared within Docker.
Procedure
From your console, navigate to PDI Service and then select the Volumes tab.
Under PDI_VOLUMES_COLLECTION, edit the volume names and add the Docker volumes you want to mount.
Declare additional mounted volumes for a PDI job
Additional volumes can be declared under the Configuration tab of the PDI Service. Any volumes you add must be declared within Docker.
Procedure
From your console, navigate to PDI Job and then select the Configuration tab.
Navigate to PDI Collection Volume Names.
This field should contain a comma-separated list of Docker volume names.Edit the volume names and add the Docker volumes you want to mount.
CautionDo not add spaces in the comma-separated list of volumes. Otherwise, the volumes and thepdi_kettle
volume fail to mount, causing the entire job to fail.
Next Step: Setting up Pentaho Worker Nodes
You can now begin the configuration steps for Setting up Pentaho Worker Nodes.