Skip to main content
Version Pentaho
Hitachi Vantara Lumada and Pentaho Documentation

Monitoring Step Performance

  1. Last updated
  2. Save as PDF

There are a few ways that you can monitor step performance in PDI.  Two tools are particularly helpful:  the Sniff Test tool and the Monitoring tab.

Sniff Test Tool

The sniff test displays data as it travels from one step to another in the stream.   To use this, complete these steps.

  1. Right-click a step in the transformation as it runs and select Sniff Test During Execution.  There are three options in this menu:
  • Sniff test input rows - Shows the data inputted into the step.
  • Sniff test output rows - Shows the data outputted from the step.
  • Sniff test error handling - Shows error handling data.
  1. After you've selected an option, values in the data stream appear.  You are also able to observe throughput. 

The sniff test is designed to be used as a supplement to logs so that you can debug complex situations. Applying a sniff test slows transformation run speed, so use with care. 

Monitoring Tab

Pentaho Data Integration provides you with a tool for tracking the performance of individual steps in a transformation. By helping you identify the slowest step in the transformation, you can fine-tune and enhance the performance of your transformations.

You enable the step performance monitoring in the Transformation Properties dialog box. To access the dialog box right-click in the workspace that is displaying your transformation and choose, Transformation Settings. You can also access this dialog box, by pressing <CTRL + T>.

TransformationProperties_Monitoring.png

As shown in the sample screen capture above, the option to track performance (Enable step performance monitoring?) is not selected by default. Step performance monitoring may cause memory consumption problems in long-running transformations. By default, a performance snapshot is taken for all the running steps every second. This is not a CPU-intensive operation and, in most instances, does not negatively impact performance unless you have many steps in a transformation or you take a lot of snapshots (several per second, for example). You can control the number of snapshots in memory by changing the default value next to Maximum number of snapshots in memory. In addition, if you run in Spoon locally you may consume a fair amount of CPU power when you update the JFreeChart graphics under the Performance tab. Running in "headless" mode (Kitchen, Pan, Pentaho Server [slave server], Carte, Pentaho BI platform, and so on) does not have this drawback and should provide you with accurate performance statistics.