Monitoring dataflows
You can monitor the status and performance of your executed dataflows to verify completed processes and maximize your resource usage. You can view details such as itemized operational metrics, execution duration, and execution status. Additionally, you can view an execution report which provides information about the time, parameters, values, and compute capacity used during processing.
The generated logging data can be sorted by level, searched by keyword, and exported for analysis. To examine the dataflow logs available to your user profile, click Monitoring from the Lumada DataOps Suite menu bar. You can also click the dataflow card to open its Dataflow summary view, where the Monitoring section provides execution information specifically for that dataflow.
Monitor a dataflow
Procedure
From the Dataflow menu bar, click Monitoring.
The Monitoring page opens, showing the status of every dataflow execution.
Use the Search and Parameters tools to help locate the dataflow that you want to view.
Use the Search box to filter through dataflow names. Use the Parameters menu to filter through the parameter names and values. See Filter by parameter for details.When you locate the dataflow that you want to view, click anywhere in its row to view the execution report.
Alternatively, in the row, you can click the More actions icon, then select View execution report.The execution report opens. See Working with execution logs for information.You can also click the More actions icon in the row, and then select Preview Parameters to see a list of related parameters and their values used during the dataflow execution.
Filter by parameter
Procedure
Click Parameters in the Monitoring page.
The Parameters dialog box opens, so you can see the parameters used by dataflow executions in the left pane and possible values of the selected parameters in the right pane.
Select the parameter and the corresponding parameter values that you want to use in your search for a dataflow execution.
Click Apply.
Results
Working with execution logs
You may want to view the execution log when troubleshooting dataflow execution issues. This log provides you with information and tools for tracking the performance of a dataflow and is part of the execution report. To learn about the sections in the execution report, see View an execution report.
The Execution Log displays at the end of the report. In addition to sorting, filtering, and searching the execution log, you can export the log file to work with later.
The following table describes the execution log features:
| Icon | Feature | Description |
 | Search field | Enter keywords into this field to search the Execution logs. |
| N/A | All lines tab | Click to show all log lines processed (number of lines in parentheses) during the execution. |
 | Errors tab |
Click to show only the log lines (number of lines in parentheses) where failures were detected during execution. Errors must be corrected before the dataflow can be executed successfully. For information about troubleshooting transformations and jobs, see Data integration issues. |
 | Warnings tab | Click to show only the log lines (number of lines in parentheses) where warnings were detected during execution. While warnings alert you to potential problems, they do not prevent successful execution. |
 | Info tab | Click to show only the log lines (number of lines in parentheses) that have processing information. |
 | Download tab | Click to export the log file. The content is exported in a ZIP file to the browser download directory. |
The execution log is comprised of the following columns:
| Column | Description |
| Level | The level of the log entry: Error, Warning, or Info. |
| Time offset | The time elapsed since the start of dataflow execution. |
| Date | The date and time the log entry was made. |
| Description | Description of the log information. |
and down 
arrows in the log table to expand and collapse additional details for each log entry. Log lines that share information gathered from similar processes are grouped together.
If the execution log has many lines, you can browse the log more quickly by using the Execution Duration slider tool. Drag the duration marker to the point in the log you want to view.
You can even use the Execution Duration slider tool while the dataflow is executing. During execution, the marker displays as white and the Live view notification displays in the upper-right corner. Use the elapsed time next to the Live view notification as a guide to position the duration marker as close as possible to the location or elasped time of the log that you want to view. When finished, click the Live view notification to return to a live view of the execution log.
When execution is complete, the duration marker is black and the label changes to Latest lines:
If you use the slider to view the earlier lines in the execution log, a Reset link displays. You can click Reset to return to the end of the log.
Export a log
Procedure
From the Dataflow menu bar, click Monitoring.
The Monitoring page opens.Locate the dataflow that you want to export, click its More actions icon, and then click the View execution report button that appears.
The execution report page opens.Click the download icon.
Results
Working with log files
You can use the log file to help identify an issue, and then take corrective action to resolve it or to fine-tune and enhance the performance of the dataflow as needed.
Download the log file as described in Export a log. Then navigate to the file, extract, and open the file to view the contents. The log entry fields in the log files are described in the following table.
| Field | Description |
| DateTimeStamp | Date and time of the log. |
| LogThreshold | The engine’s internal logging level (BASIC, DETAILED, DEBUG, ROWLEVEL, MINIMAL, and ERROR). |
| Message | Log message. |
The following table maps the internal logging levels of the engine to the corresponding logging levels of the execution report.
| Engine's internal logging level | Dataflow execution report |
| BASIC | INFO |
| DETAILED | INFO |
| DEBUG | INFO |
| ROWLEVEL | INFO |
| MINIMAL | WARN |
| ERROR | ERROR |
You can use log entries to understand the individual processes occurring during the execution of your dataflow. Take a look at the following example of a log entry in a log file:
2020-11-16 13:40:23.331 Z [BASIC]: Step Write to log.0 ended successfully, processed 1 lines. ( 0 lines/s)
You can see the time the log entry was made, the threshold of the logging level, the message about the operation being performed, and the results of that process, as mapped in the following table:
| Field | Description |
| DateTimeStamp | 2020-11-16 13:40:23.331 Z (where Z is time zone Zulu, UTC+0) |
| LogThreshold | BASIC |
| Message | Step Write to log.0 ended successfully, processed 1 lines. ( 0 lines/s) |