Row Denormaliser
The Row Denormaliser step can be used to try to improve performance either by adding redundant copies of data values or by grouping data. You can also use this step to convert data types.
The data must be normalized before you can denormalize it.
For example, consider the following set of normalized data:
| Produce Category | Delivery Time | Produce Type |
|---|---|---|
| fruit |
2018/01/05 08:00:00 |
apples |
|
fruit |
2018/01/05 08:10:00 | oranges |
| fruit | 2018/01/05 08:20:00 | apples |
| fruit | 2018/01/05 08:30:00 | oranges |
The data can be denormalized by grouping on the type of fruit, as shown in the following table:
| Produce Category | Apples | Oranges |
|---|---|---|
| fruit |
2018/01/05 08:00:00 |
2018/01/05 08:10:00 |
| fruit |
2018/01/05 08:20:00 |
2018/01/05 08:30:00 |
General
Enter the following information in the transformation step fields.
| Field | Description |
|---|---|
| Step name |
Specify the unique name of the transformation step on the canvas. The step name is set to 'Row denormaliser' by default. |
| Key field |
Define the key of the output row. The available fields are from the incoming PDI data stream. Another step with output data must be connected to the Row Denormaliser step via hop for options to appear in this field. |
Group Field Table
Use The fields that make up the grouping table to specify what fields from the incoming PDI data stream define the grouping for the denormalization process. You can click Get Fields to automatically fill the table with all the fields from the incoming PDI data stream.
Target Fields Table
Use the Target fields table to select the fields to denormalize by specifying the String value for the Key field. Options are provided to convert data types. Strings are most common as key-value pairs so you must often convert to Integer, Number or Date. If you get key-value pair collisions (key is not unique for the group specified) specify the aggregation method to use. You can click Get lookup fields to retrieve fields from the PDI data stream.
Examples
The data-integration/samples/transformations directory contains the following example KTRs that show how to use this step:
| Example KTR File | Description |
|---|---|
| Denormaliser – Simple Example.ktr | Converts a ‘month’ row in a record to a multiple ‘month’ columns. |
| Denormaliser – Data Conversion and Aggregation.ktr | Converts data values by ‘month’. |
| Denormaliser – 2 Series of Key-Value Pairs.ktr | Converts ‘name’ and ‘ID’ rows to multiple ‘name’ and ‘ID’ columns. |
| Denormaliser – Capturing Last State Timestamp.ktr | Aggregates data values by ‘state’. |
Metadata Injection Support
All fields of this step support metadata injection. You can use this step with ETL Metadata Injection to pass metadata to your transformation at runtime.