Executing dataflows

Execute a dataflow

Perform the following steps to execute a dataflow:

Procedure

From the Card view, click Execute on the dataflow card you want to run.

NoteClicking anywhere on the card opens the Summary page, where you can click Execute.
The Execution Settings page opens. You can execute a dataflow from the Execution Settings page after you set additional values, or you can set schedules for your dataflow. See Scheduling dataflows for more information.
(Optional) Modify the parameter values you want to use for the execution.
See Edit a dataflow parameter for details.
Click Execute in the upper-right corner of the Execution Settings page to execute the dataflow.

Results

The dataflow executes with the specified parameters.

Next steps

If you need to make small changes to your dataflow, open the Lumada Data Transformation Editor to perform light editing tasks. See Editing dataflows for more information.

Edit a dataflow parameter

Depending on your assigned roles and permissions, you can use parameters to customize dataflows. Parameters are created in PDI and are imported with your dataflow. You cannot add or remove dataflow parameters, but you can modify the default parameter values.

You can also edit how you interact with your parameters. When parameters are imported with dataflows into Lumada DataOps Suite, they default to the Text type. Changing the parameter type allows you to execute a dataflow with prompts for other information types. You can edit parameters to change the prompt to one of the following types:

Text
Checkbox
Single selection
Multiple selection
Date

Perform the following steps to edit the parameters of a dataflow:

Procedure

On the Card view, click View on the dataflow card you want to modify.

NoteClicking anywhere on the card opens the Summary page.
The Summary page opens.
(Optional) Click the Visibility icon on the Summary page if you want to hide the associated parameter.
Hiding the parameter prevents a user from changing the value of that parameter. The parameter does not appear on the Execution Settings page, but does appear on the Execution report.
Click the Pencil icon next to the parameter that you want to edit.
The Edit parameter dialog box opens.

NoteDisplay name and Prompt type are required properties.

Next steps

You can also prevent a Text prompt from appearing in the Execution report. Single selection and Multiple selection prompt types display selections or multiple check boxes, depending on the number of parameter values you assign as shown in the following table:

Number of values	Display
1-3	A radio button group for a single selection, or a check box for multiple selections.
4-9	Drop-down menu.
10 or more	Drop-down menu with search.

You can mark Text prompt values as required. Values for Single selection prompts are always required.

View an execution report

Viewing an execution report can help you identify possible issues with a dataflow. To view a report of the latest dataflow execution, click the More actions icon in the bottom-right corner of the dataflow card, then select View execution report.

View execution report option

You can also click the More actions icon on the row for the dataflow on the Monitoring page and click View execution report.

The report is divided into several sections, which are accessible from the left navigation pane:

Overview
A summary that includes the status of the execution, user, and elapsed times for the steps or entries processed.

Execution Parameters
List of execution parameters, if any, and their values.

Advanced Settings
Advanced settings and their values, such as the selected Compute capacity and Log level. To learn more about changing these values, see Advanced settings.

Resources
Shows average, peak, and allocated CPU and Memory resources used by the dataflow. To learn more about changing these values, see Advanced settings.

Execution Logs
Displays line-by-line processing information and total occurrences for each of 3 log levels: Errors, Warnings, and Info. You can filter these results and examine the processed data to determine if issues were encountered. For example, if a step or entry fails, you can view the error with a description of the failure. You can also export a log file for stack trace analysis. See Working with execution logs for more information.

Dataflow execution report

See Monitoring dataflows to view reports of previous executions.

Schedule an execution

You can schedule the execution of a dataflow on the Execution Settings page.

Learn more

Advanced settings

You can modify the advanced settings for customizations of the compute capacity and for the logging level used for dataflow executions.

Setting the compute capacity

You have control over your computing resources so you can quickly scale capacity as your computing requirements change. If needed, your system administrator can also adjust the number of Central Processing Units (CPUs) and memory associated with each compute capacity option. Use the following table to determine your compute capacity allocations.

Compute capacity	Description
Small	1 CPU and 2048 MB memory
Medium (Default)	1.5 CPUs and 3072 MB memory
Large	2 CPUs and 4096 MB memory

Set the compute capacity

Follow the steps below to set the compute capacity:

Procedure

On the Flows page, click Execute on the dataflow card.
Alternatively, you can modify the compute capacity of a scheduled execution from the Schedules page. Cick Edit in the More options menu for the schedule you want to modify.The Execution Settings page opens.
Turn on the toggle under Advanced settings to view the options.
Click Compute capacity.
From the drop-down menu, select the size you want to use for dataflow execution.

Setting the log level

You have the following logging level options to display as little or as much information as you need, so you can capture the right amount of information needed to monitor, analyze, and understand the execution of your dataflow.

All lines in the log are preceded by the date and time of day. Debug and Row Level logging levels contain information which may be too sensitive to be shown to all your users, so you may want to consider the sensitivity of your data when selecting these logging levels. See Monitoring system performance to understand how best to use these logging methods.

The logging levels include:

Log Level	Description
Nothing	No logging occurs.
Error	Only errors are logged.
Basic	Provides the standard logging output (default).
Detailed	Provides a detailed logging output.
Debug	For debugging purposes, provides very detailed output.
Row Level	Logging at the most detailed level. This log level generates a lot of log data.

Set the log level

Follow the steps below to set the log level:

Procedure

On the Flows page, click Execute on the dataflow card.
Alternatively, you can modify the log level from the Schedules page. Click Edit in the More options menu for the schedule you want to modify.The Execution Settings page displays.
Turn on the toggle under Advanced settings to view the options.
Click Log level.
From the drop-down menu, select the setting for the amount of detail you want in the execution log.