PDI Steps Optimized for AEL-Spark
This table lists PDI transformation steps specifically coded to the Spark APIs for optimized performance with the Adaptive Execution Layer. The table also indicates which PDI version introduced the optimized step.
The dagger symbol ( † ) indicates steps with known AEL considerations. For a complete list, see PDI Steps with AEL Considerations.
| Step | PDI Version |
|---|---|
| Abort | 8.1 |
| Avro Input† | 8.0 |
| Avro Output† | 8.0 |
| AMQP Consumer | 8.1 |
| Copy Rows to Result | 8.1 |
| ETL Metadata Injection | 8.2 |
| Filter Rows | 7.1 |
| Get Records from Stream | 7.1 |
| Group By | 8.1 |
| Hadoop File Input† | 7.1 |
| Hadoop File Output† | 7.1 |
| Hbase Input† | 8.0 |
| Hbase Output† | 8.1 |
| Kafka Consumer | 8.1 |
| Mapping (Sub-transformation) | 8.2 |
| Mapping Input Specification | 8.2 |
| Mapping Output Specification | 8.2 |
| Memory Group By | 8.1 |
| Merge Join | 8.1 |
| MQTT Consumer | 8.1 |
| ORC Input† | 8.1 |
| ORC Output† | 8.1 |
| Parquet Input† | 8.0 |
| Parquet Output† | 8.0 |
| Simple Mapping | 8.2 |
| Sort Rows | 7.1 |
| Stream Lookup | 7.1 |
| Table Input† | 8.2 |
| Table Output | 8.2 |
| Text File Input† | 7.1 |
| Text File Output† | 7.1 |
| Transformation Executor† | 8.1 |
| Unique Rows | 8.1 |
| Unique Rows (Hashset) | 8.1 |
| Write to Log | 8.2 |