Skip to main content
Hitachi Vantara Lumada and Pentaho Documentation

Introduction to Streamlined Data Refinery


Create a single data refinery by streamlining all of your data sources through a central processing hub.


The Streamlined Data Refinery (SDR)  is a simplified and specific ETL refinery composed of a series of Pentaho Data Integration (PDI) jobs that take raw data, augment and blend it through the request form, and then publish it for report designers to use in Analyzer.

We have created the Movie Ratings-SDR sample for example purposes to help you get familiiar with the SDR structure. The sample described here was developed by Pentaho and is based on CTools.

How Does the SDR Work?

The components that make up the data refinery are PDI, used for parameter entry, working in conjunction with an app for refining the data. This app calls to the Data Integration (DI) Server for the main job: refining data through Spoon using the new Build Model job entry, then publishing the data source back to the Business Analytics (BA) Server through the Publish Model job entry. Once it is published, the refined data is available for use in creating Analyzer reports. This process is shown in the graphic.

SDR Workflow

App Builder, Community Dashboard Editor, and CTools

App Builder is an application builder for people who may not have Java knowledge, but who may have plenty of interesting ideas for new plugins. All that is required to use App Builder is knowledge of CTools and PDI.

Community Dashboard Editor (CDE), when integrated with the Pentaho User Console (PUC), simplifies the process of creating, refining, and previewing Pentaho dashboards. You can use CDE to design dashboards, either from scratch or using a template. 

Learn More about Our Tools

Here are a few ways to find out more about the Streamlined Data Refinery and other Pentaho products.