Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com

 

Hitachi Vantara Lumada and Pentaho Documentation

Managing configurations

Parent article

In Lumada Data Catalog, you can change specific configuration properties that are used by scripts when gathering catalog metadata, tags, and tag associations.

To update configuration properties, navigate to Manage and click Configuration. Many of these properties require a server restart, which you can perform in the configuration window.

The configuration controls are organized into sections for the Lumada Data Catalog Metadata Server (metadata-server), the Lumada Data Catalog Agent (local-agent), and the Lumada Data Catalog Application Server (app-server) components. If you have created multiple agents, then a configuration section for each agent is displayed. Each component has its own restart button.

Configuration categories

The LDC Metadata Server, LDC Agent, and LDC Application Server all have the following configuration categories available:

  • MetadataService

    Manage metadata service properties.

  • DISCOVERY

    Manage discovery related properties.

  • JobManager

    Manage job manager configuration properties.

  • DISCOVERY_PROFILER

    Manage profile discovery related configuration properties.

  • MISC

    Manage miscellaneous control properties.

  • SECURITY

    Manage security related configuration properties.

  • API

    Manage API related properties. Other API properties may be available but are not listed here. For other properties, see the Hitachi Vantara Lumada and Pentaho Support Portal.

Users assigned the administrator role can configure most properties in the Configuration page, but some properties are unavailable for editing. If the value set for the property is different than the default value, the property appears in bold text.

The following image shows the Discovery configuration properties. Note that the default value for the ldc.discovery.export_to_csv.enabled property has been changed from false to true so applicable users can now export Discovery results to CSV. Export to CSV configuration option

After you make select configuration changes, you must restart the associated component (LDC Application Server, LDC Metadata Server, or LDC Agent). If the configuration changes affect more than one component, the components need to be restarted in a specified order. See Restart a server.

Searching configurations

You can search the Configuration page for a specific property that you want to update. The Search configuration options box accepts keyword searches, so even if you do not know the exact label for the configuration property you want, a keyword triggers a list of possible matches for the keyword you entered.

In the example below, a search for "notification" results in 2 matches for each server:GUID-4CAA3810-DC52-4BB3-BA27-6809167DEA8F-low.png

Server restarts

After you update select configuration properties, you must restart the server. These configuration properties are indicated with an asterisk (*) as shown in the following image.

Server restart asterisks on properties

After you update a configuration property that requires a server restart, return to the Configuration page and click the restart button for the server you are working on, either Restart app-server, Restart metadata-server, or Restart local-agent.Metadata Server restart button

Restart the components in the following order:

  1. LDC Application Server
  2. LDC Metadata Server
  3. LDC Agent

For example, if you need to restart the LDC Application Server, you should then restart the LDC Metadata Server and the LDC Agent in that order. If the configuration is changed in the LDC Metadata Server and LDC Agent, you need to restart the LDC Metadata Server first and then the LDC Agent. As a best practice, you should verify that the LDC Metadata Server is up before you restart the LDC Agent.

To verify that the LDC Metadata Server is up, go to Manage then click Network and look for a check mark in the Status column for the LDC Metadata Server. When you hover the cursor over the check mark, the status Metadata server connected appears.

To check the status of the LDC Agent, go to Manage then click Agents. If the LDC Agent is up, a green circle appears in the Connected column. Go to Manage then click Network and look for a check mark in the Status column for the LDC agent server. When you hover the cursor over the check mark, the following status appears:

1. Appserver connected. 2. Metadata server connected.

Restart a server

If a configuration property requires that you restart the server after setting the property, you must restart the components in a specific order for Data Catalog to continue to run smoothly. You should restart the components in the following order:

  1. LDC Application Server
  2. LDC Metadata Server
  3. LDC Agent

Perform the following steps to restart the components in the proper order:

Procedure

  1. Navigate to Manage and then click Configuration, then click the restart button applicable to the component you need to restart.

    If you changed a configuration property on the LDC Application Server, click Restart app-server to restart the LDC Application Server.
  2. Click Restart meta-server to restart the LDC Metadata Server.

  3. Go to Manage and then click Network to check the status of the LDC Metadata Server. The Status column should have a check mark. Hover over the check mark and ensure the following status appears: Metadata server connected.

  4. Click Manage and then Configuration and click Restart local-agent to restart the LDC Agent.

  5. Click Manage then click Agents to check the status of the LDC Agent.

    The Connected column for the LDC Agent should have a green circle.
  6. Go to Manage and then click Network.

    In the Status column for the LDC Agent, there should be a check mark. Hover over the check mark and confirm the status is the following:
    1. Appserver connected.
    2. Metadata server connected.

Setting configuration properties

Select configuration properties for the LDC Application Server, LDC Agent, and LDC Metadata Server may allow for smooth functioning of your Data Catalog environment. You can set these properties in the configuration window.

AttentionThe "Setting of properties in the configuration window" feature is available in version 6.1 and later.

Notifications

Notifications are essential for collaboration between key contributors or stakeholders. They are designed to capture a user's attention for tasks or posts that require their input or acknowledgement.

In Data Catalog, notifications are triggered in the following scenarios:

  • Job completion
  • Posts
  • Ask experts
  • Virtual folder changes
  • Share resources

These notifications appear in the Data Catalog as an incremental count badge on the Bell icon in the dashboard menu bar.

The two types of notification configurations include:

  • Notification configuration for enabling or disabling email notifications.
  • Notification configuration for job events.

Set up email notifications

You can use the Data Catalog user interface to set up notification triggers for email notifications. When configured, email notifications are sent as the Data Catalog service user.

Perform the following steps to enable email notifications:

Procedure

  1. As the Data Catalog service user, enter the following required details in the LDC-HOME/app-server/conf/notification-config.json file to configure outgoing SMTP server settings:

    {
        "starttls": true,
        "host": "smtp.office365.com",
        "port": 587,
        "auth": true,
        "user": "<data catalog service user email id>",
        "password": "<encrypted-password>"
    }              
    NoteOnly the Data Catalog service user can update this configuration in the LDC Application Server's CLI.
  2. Enter the following command to encrypt the email password:

    APP-SERVER-HOME$ bin/ldc-util encrypt
  3. To ensure that the notification insights tool works correctly, you need to restart the components in the following order:

    1. Application Server
    2. Metadata Server
    3. Agent

    For more information, see Restart a server.

  4. Navigate to Manage and then click Configuration.

  5. Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.

  6. Expand the MISC section.

  7. Expand the Send Email Notifications property.

  8. Set the switch to true and click Save.

Set up job event notifications

By default, notifications are generated for the following job status changes:
  • Submitted
  • Success
  • Failed
  • Cancelled
  • Skipped

You can configure which status changes generate notifications by updating the Notification Trigger Points configuration property.

Perform the following steps to set up job event notifications:

Procedure

  1. Navigate to Manage and then click Configuration.

  2. Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.

  3. Expand the MISC section.

  4. Expand the Notification Trigger Points property.

  5. Edit the list of job events to set the job events you want to trigger notifications.

    For example, you can add incomplete to the comma separated list of default values to trigger notifications when a job is incomplete. You can also click the refresh icon to reset the default values.
  6. Click Save.

Set up job concurrency

In Data Catalog you can run jobs concurrently for virtual folders without overlapping paths by using Apache ZooKeeper. ZooKeeper validates the path overlap conditions and sets read/write locks on the source paths in the path specifications of the virtual folder.

Before performing this task, you should have a Znode path that is writable by the Data Catalog service user.

Perform the following steps to set up job concurrency:

Procedure

  1. Navigate to Manage and then click Configuration.

  2. Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.

  3. Expand the DISCOVERY section.

  4. Expand the Zookeeper Locking for Job Concurrency property.

  5. Set the switch to true and click Save.

  6. Expand the ZooKeeper Locking Base Znode property.

  7. Enter the znode base path used for locking. The path must exist and must be writable by the Data Catalog service user.

    For example, zkhost1:2181,zkhost2:2181,zkhost3:2181/.
  8. Click Save.

Set up custom headers and footers

With Data Catalog you can create custom headers and footers. The header-footer configuration is controlled by the custom-header.properties file found in the APP-SERVER-HOME/conf path.

Perform the following steps to customize the headers and footers:

Procedure

  1. From the command prompt, open the custom-header.properties file for editing using the following command:

    APP-SERVER-HOME$ vi conf/custom-header.properties
  2. Add your customizations.

    For example, you may customize font size or font color for the messages in the header and footer.
  3. When finished, save and exit.

    NoteA Jetty restart is not required for these configuration changes to take effect.

    The following image shows the web application with customizations:

    Customized example

Customize a login screen message

You can place a custom message on the login screen to identify your organization or specific department, or to have an inspirational quote of the day. You can set up your custom message in the Data Catalog user interface.

Perform the following steps to customize a login screen message:

Procedure

  1. Navigate to Manage and then click Configuration.

  2. Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.

  3. Expand the MISC section.

  4. Expand the Custom login message enabled property.

  5. Set the switch to true and click Save.

  6. Expand the Custom message to display on the login screen property.

  7. Enter your custom message and click Save.

  8. Return to the Configuration page and click the applicable Restart button to restart the server or agent.

Set up most frequent values and patterns

In Data Catalog, you can set the properties that display the top and bottom most frequent sample values and patterns of frequency distributions. You can change these settings in the Data Catalog configuration window.

Perform the following steps to set up the most frequent values and patterns:

Procedure

  1. Navigate to Manage and then click Configuration.

  2. Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.

  3. Expand the MISC section.

  4. Expand the Number of top and bottom patterns count property.

  5. Enter an integer to be split between top and bottom patterns and click Save.

    By default, this value is set to 6, which is split into 3 top and 3 bottom patterns.
  6. Return to the Configuration page and expand DISCOVERY_PROFILER in the applicable section.

  7. Expand the Memory multiplier for most frequent values and patterns property.

  8. Enter an integer for memory slots for most frequent values and most frequent patterns to use during profiling.

  9. Click Save.