Administering the DataOps Suite

Open the Lumada DataOps Suite and log in with your assigned admin credentials. If you do not have these credentials, see the IT administrator who installed and configured your system or your Hitachi Vantara Customer Success representative.

After you log in, the Home page opens. Click the App Switcher icon in the upper-right of the window, and select Solution management from the Applications menu that appears.

The Solution management window is only available to users assigned admin permissions.

Under Solutions in the navigation pane, select Installed. The canvas displays the cards for your installed solutions.

In addition to cards for your main DataOps components, such as Lumada Data Catalog and Lumada Dataflow Studio, you can also view cards for the components that control your DataOps displays and backend services. You can view the configuration and monitor the health of all your solutions from this single dashboard.

Under Solutions in the navigation pane, select Available. In the canvas, browse the list of deployed solutions available for install or upgrade. To find a specific solution to install or upgrade, click a column header to sort the column information, and then click the solution name to install or upgrade it.

Lumada DataOps Suite is shipped with the following installed solutions:

• Control Plane

  Controls the user interface for LDOS, including branding (logos, labels,etc.), navigation, the App Switcher, notifications, and user preferences.

• Dataflow Engine

  Manages the ETL engine for dataflows in a cluster environment.

• Dataflow Engine Broker

  Connects the engine to Dataflow Studio.

• Data Transformation Editor

  Use to lightly edit PDI client transformation (.ktr) and job (.kjb) files.

• Dataflow Importer

  Imports transformation (.ktr) and job (.kjb) files from Data Transformation Editor and the PDI client as dataflows into Dataflow Studio.

• Dataflow Studio

  Manages dataflows, including executing, monitoring, and scheduling activities.

• Lumada Data Catalog

  Application that automates discovery, classification, and mangement of your organization's data.

• Messaging Service

  Configures the messaging service for communication.

• Object Storage Service

  Storage server shipped with LDOS to support Data Catalog objects.

  ImportantThis server is not supported in production use. As a best practice, your organization should point to where you are securely storing data. Ask your IT administrator or Hitachi Vantara Customer Success representative for more information.

• Solution Control Plane

  Controls the user interface and services of the Solution management application. In addtion, the Solution Conrol Plane provides access to the Identity and Access Management console and to logging information.

Each card on the canvas provides information about the solution or service it performs in the Lumada DataOps Suite. Click a card to view its details.

Each card features a unique solution name that was created at the time of installation. Additionally, each card contains the version number of the solution, its current health status, and a brief description. Drill-down into the following tabs for more detailed information:

• Status

  View the overall health of the solution or service, which is a combination of both installation and resource health. Resource health is derived from the health of individual resources. See Monitoring solution health for more information.

• Resources

  View a list of your running, pending, and successful resource executions.

• Applications

  Provides links to the applications comprising the solution. These links may connect to services or to informational resources, such as API reference guides.

• Configuration

  Validate or edit the values.yaml file for the configuration of the selected solution or service.

  NoteDo not change the value of the parameters in the values.yaml file for the Solution Control Plane. For guidance, consult your IT administrator or your Hitachi Vantara Customer Success respresentative.

• History

  Lists the installed versions and any changes or upgrades that have been made to the initial configuration of a solution. For example, you can view the curently deployed version and all the previously installed solutions which are now superseded by the currently deployed version. You can roll back an upgraded or revised solution or service to a previous version, provided that all software dependencies have been satisfied. See Rollback a revision.

As the DataOps Suite administrator, you can configure installed solutions to optimize health and performance for your organization. Configuration settings are located in a YAML file for each solution and should be configured according to your organization's settings for that application or service. See the Solutions configuration reference to configure your individual solutions.

To access the configuration settings for an individual solution, open Solution management and navigate to the installed solution. Click the Configuration tab for the selected solution.

For identity and access management, you can specify users and permissions to the applications and secure services provided in Lumada DataOps Suite. By default, as the LDOS administrator, you can access the Keycloak Default Realm Console to manage user and permissions settings. Keycloak provides LDOS authentication and integration with LDAP.

Roles are part of a global namespace, which is shared and manages a set of users, credentials, roles, and groups across all LDOS solutions. Administrator, Analyst, and Data Scientist are some examples of the default realm roles available to your organization. The role mappings you define set the permission types between a role and a user. You can associate a user with one or more roles, or none at all. For ease of management, you may want to assign access and permissions to specific roles rather than individual users, which can be difficult as the number of users changes over time.

To help you get started, Lumada DataOps Suite is shipped with the following default roles and sample users. Each default role and sample user combination is assigned a standard set of permissions, which correspond to roles and permissions in LDOS solutions and tools.

Lumada DataOps Suite also provides each client with its own dedicated namespace where roles can be defined. Clients are entities that can request user authentication. Clients can be applications and services that want to use secure sign-on. They can also be entities that want to request identity information or to securely call other LDOS services. You can use client roles to further define role actions for specific users, and map those roles to actions authorized in the corresponding solutions. See the Keycloak documentation for more information.

You can also create composite roles that have one or more additional role associations. For example, Data Engineer and Analyst are composite roles that are made of client-specific roles originating from multiple solutions, including Data Catalog and the Control Plane.

As an LDOS administrator, you can monitor the health of all solutions in the Installed view. Each card in the canvas displays color-coding and an icon to indicate the status of the solution.

A solution's health status is determined by a combination of its installation health and its resource health. The following table defines the possible statuses with their assigned icons:

Status	Icon	Color	Description
Healthy		Green	The solution is installed and running with healthy resources.
Warning		Yellow	The solution has issues that may impact performance, such as a health issue with component resources.
Failed		Red	The solution is unable to be deployed because of major failures requiring administrator attention.
Unknown	None	Gray	Health cannot be determined.

Click a card to inspect the specific health information for a solution. You can view the installation health and resource health in the Status tab of a solution's card to locate an issue to resolve.

• Installation Health

  The Installation Health card represents the current release status for the solution, which can be one of the following:

  • Deployed (healthy)
  • Warning (includes states such as Uninstalling, Pending-install, Pending-rollback, and Pending-upgrade)
  • Failed
  • Unknown
  To inspect previous installations, click the Installation Health card to open the History tab.

• Resource Health

  The Resource Health card represents the collective health of the running processes of the Kubernetes pods for the solution. To inspect the health of individual pods, click the Resource Health card to open the Resources tab.

  

In the Resources tab, you can inspect the list of health statuses for each pod. For example, you can see that a pod may be running, but with errors such as a high restart count. Statuses include:

Status	Icon	Description
Healthy		All containers are ready, or the job is complete.
Warning		Some but not all containers are ready, or a job is in a non-healthy but non-failed state.
Failed		No containers are ready, or the job has failed.
Unknown	None	No containers have been defined.