Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com. 

 
Version Data Integration and Analytics
Hitachi Vantara Lumada and Pentaho Documentation

Single Threader

  1. Last updated
  2. Save as PDF
 

Parent article

The Single Threader step executes a sequence of steps in a single thread, which allows you to tune the performance of your transformations by limiting the number of threads or processors used:

  • To solve issues in transformations with many steps by reducing the data passing and thread context switching overhead.
  • To sort data rows in a specific time frame or by a specific number for real-time streaming.
  • To process chunks of data, pause, and then continue processing after reaching the chunk size.

Related information: White Paper on Single Threading in Pentaho Data Integration

Parent Topic
Child Topics

General

 

Enter the following information in the transformation step fields:

Field Description
Step Name The unique name of the step on the canvas. A transformation step can be placed on the canvas several times; however, it represents the same transformation step. You can customize the name or leave it as the default (Single Threader).
Transformation

Specify your sub transformation for single threading by entering in its path or clicking Browse.

If you select a transformation that has the same root path as the current transformation, the variable ${Internal.Entry.Current.Directory} will automatically be inserted in place of the common root path. For example, if the current transformation's path is /home/admin/transformation.ktr and you select a transformation in the folder /home/admin/path/sub.ktr then the path will automatically be converted to ${Internal.Entry.Current.Directory}/path/sub.ktr.

If you are working with a repository, specify the name of the transformation in your repository. If you are not working with a repository, specify the XML file name of the transformation on your system.

Transformations previously specified by reference are automatically converted to be specified by name within the Pentaho Repository.
Parent Topic

Options

 

The Single Threader step features two tabs with fields. Each tab is described below.

Parent Topic
Child Topics

Options tab

 

Single Threader

This tab contains the following fields:

Option Description
Injector step Specify the step that that satisfies the Mapping Input Specifications (filename, type, length, and precision) by entering its name or clicking Get.
Retrieval step Specify the step from which the data is retrieved by entering its name or clicking Get.
Batch size Specify when processing of a chunk of data starts based on the number of rows received, which is 100 by default. Alternately, specify Batch time (ms) to process based on a time delay.
Batch time (ms) Specify when processing of a chunk of data starts based on the specified time delay, instead of Batch size.
Parent Topic

Parameters tab

 

Parameters tab in Single Threader

Enter the following information to pass parameters to the transformation:

Option Description
Pass parameter values to sub transformation Pass all parameters of the transformation down to the sub-transformation.
Parameter name Specify the parameter name passed to the sub-transformation.
Fixed value

Specify sub-transformation parameter values through one of the following actions:

  • Enter a value manually (Sub transformation, for example).
  • Use a parameter to set the value (${Internal.Job.Name}, for example).
  • Use a combination of manually specified values and parameter values (${FILE_PREFIX}_${FILE_DATE}.txt, for example).
Get Parameters Get the existing parameters associated with the sub-transformation.
Parent Topic