Managing configurations
In Lumada Data Catalog, you can change specific configuration properties that are used by scripts when gathering catalog metadata, tags, and tag associations.
To update configuration properties, navigate to Manage and click Configuration. Many of these properties require a server restart, which you can perform in the configuration window.
The configuration controls are organized into sections for the Lumada Data Catalog Metadata Server (metadata-server), the Lumada Data Catalog Agent (local-agent), and the Lumada Data Catalog Application Server (app-server) components. If you have created multiple agents, then a configuration section for each agent is displayed. Each component has its own restart button.
The LDC Metadata Server, LDC Agent, and LDC Application Server all have the following configuration categories available:
MetadataService
Manage metadata service properties.
DISCOVERY
Manage discovery related properties.
JobManager
Manage job manager configuration properties.
DISCOVERY_PROFILER
Manage profile discovery related configuration properties.
MISC
Manage miscellaneous control properties.
SECURITY
Manage security related configuration properties.
API
Manage API related properties. Other API properties may be available but are not listed here. For other properties, see the Hitachi Vantara Lumada and Pentaho Support Portal.
Users assigned the administrator role can configure most properties in the Configuration page, but some properties are unavailable for editing. If the value set for the property is different than the default value, the property appears in bold text.
The following image shows the Discovery configuration properties. Note that the
default value for the ldc.discovery.export_to_csv.enabled
property has been
changed from false to true so applicable
users can now export Discovery results to CSV.
After you make select configuration changes, you must restart the associated component (LDC Application Server, LDC Metadata Server, or LDC Agent). If the configuration changes affect more than one component, the components need to be restarted in a specified order. See Restart a server.
Searching configurations
You can search the Configuration page for a specific property that you want to update. The Search configuration options box accepts keyword searches, so even if you do not know the exact label for the configuration property you want, a keyword triggers a list of possible matches for the keyword you entered.
In the example below, a search for "notification" results in 2 matches for each server:
Server restarts
After you update select configuration properties, you must restart the server. These configuration properties are indicated with an asterisk (*) as shown in the following image.
After you update a configuration property that requires a server restart, return to the Configuration page and click the restart button for the server you are working on, either Restart app-server, Restart metadata-server, or Restart local-agent.
Restart the components in the following order:
- LDC Application Server
- LDC Metadata Server
- LDC Agent
For example, if you need to restart the LDC Application Server, you should then restart the LDC Metadata Server and the LDC Agent in that order. If the configuration is changed in the LDC Metadata Server and LDC Agent, you need to restart the LDC Metadata Server first and then the LDC Agent. As a best practice, you should verify that the LDC Metadata Server is up before you restart the LDC Agent.
To verify that the LDC Metadata Server is up, go to Manage then click Network and look for a check mark in the Status column for the LDC Metadata Server. When you hover the cursor over the check mark, the status Metadata server connected appears.
To check the status of the LDC Agent, go to Manage then click Agents. If the LDC Agent is up, a green circle appears in the Connected column. Go to Manage then click Network and look for a check mark in the Status column for the LDC agent server. When you hover the cursor over the check mark, the following status appears:
Restart a server
If a configuration property requires that you restart the server after setting the property, you must restart the components in a specific order for Data Catalog to continue to run smoothly. You should restart the components in the following order:
- LDC Application Server
- LDC Metadata Server
- LDC Agent
Perform the following steps to restart the components in the proper order:
Procedure
Navigate to Manage and then click Configuration, then click the restart button applicable to the component you need to restart.
If you changed a configuration property on the LDC Application Server, click Restart app-server to restart the LDC Application Server.Click Restart meta-server to restart the LDC Metadata Server.
Go to Manage and then click Network to check the status of the LDC Metadata Server. The Status column should have a check mark. Hover over the check mark and ensure the following status appears: Metadata server connected.
Click Manage and then Configuration and click Restart local-agent to restart the LDC Agent.
Click Manage then click Agents to check the status of the LDC Agent.
The Connected column for the LDC Agent should have a green circle.Go to Manage and then click Network.
In the Status column for the LDC Agent, there should be a check mark. Hover over the check mark and confirm the status is the following:1. Appserver connected. 2. Metadata server connected.
Setting configuration properties
Select configuration properties for the LDC Application Server, LDC Agent, and LDC Metadata Server may allow for smooth functioning of your Data Catalog environment. You can set these properties in the configuration window.
Notifications
Notifications are essential for collaboration between key contributors or stakeholders. They are designed to capture a user's attention for tasks or posts that require their input or acknowledgement.
In Data Catalog, notifications are triggered in the following scenarios:
- Job completion
- Posts
- Ask experts
- Virtual folder changes
- Share resources
These notifications appear in the Data Catalog as an incremental count badge on the Bell icon in the dashboard menu bar.
The two types of notification configurations include:
- Notification configuration for enabling or disabling email notifications.
- Notification configuration for job events.
Set up email notifications
Perform the following steps to enable email notifications:
Procedure
As the Data Catalog service user, enter the following required details in the
LDC-HOME/app-server/conf/notification-config.json
file to configure outgoing SMTP server settings:{ "starttls": true, "host": "smtp.office365.com", "port": 587, "auth": true, "user": "<data catalog service user email id>", "password": "<encrypted-password>" }
NoteOnly the Data Catalog service user can update this configuration in the LDC Application Server's CLI.Enter the following command to encrypt the email password:
APP-SERVER-HOME$ bin/ldc-util encrypt
To ensure that the notification insights tool works correctly, you need to restart the components in the following order:
- Application Server
- Metadata Server
- Agent
For more information, see Restart a server.
Navigate to Manage and then click Configuration.
Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.
Expand the MISC section.
Expand the Send Email Notifications property.
Set the switch to true and click Save.
Set up job event notifications
- Submitted
- Success
- Failed
- Cancelled
- Skipped
You can configure which status changes generate notifications by updating the Notification Trigger Points configuration property.
Perform the following steps to set up job event notifications:
Procedure
Navigate to Manage and then click Configuration.
Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.
Expand the MISC section.
Expand the Notification Trigger Points property.
Edit the list of job events to set the job events you want to trigger notifications.
For example, you can addincomplete
to the comma separated list of default values to trigger notifications when a job is incomplete. You can also click the refresh icon to reset the default values.Click Save.
Set up job concurrency
Before performing this task, you should have a Znode path that is writable by the Data Catalog service user.
Perform the following steps to set up job concurrency:
Procedure
Navigate to Manage and then click Configuration.
Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.
Expand the DISCOVERY section.
Expand the Zookeeper Locking for Job Concurrency property.
Set the switch to true and click Save.
Expand the ZooKeeper Locking Base Znode property.
Enter the znode base path used for locking. The path must exist and must be writable by the Data Catalog service user.
For example,zkhost1:2181,zkhost2:2181,zkhost3:2181/
.Click Save.
Set up custom headers and footers
Perform the following steps to customize the headers and footers:
Procedure
From the command prompt, open the custom-header.properties file for editing using the following command:
APP-SERVER-HOME$ vi conf/custom-header.propertiesAdd your customizations.
For example, you may customize font size or font color for the messages in the header and footer.When finished, save and exit.
NoteA Jetty restart is not required for these configuration changes to take effect.The following image shows the web application with customizations:
Customize a login screen message
Perform the following steps to customize a login screen message:
Procedure
Navigate to Manage and then click Configuration.
Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.
Expand the MISC section.
Expand the Custom login message enabled property.
Set the switch to true and click Save.
Expand the Custom message to display on the login screen property.
Enter your custom message and click Save.
Return to the Configuration page and click the applicable Restart button to restart the server or agent.
Set up most frequent values and patterns
In Data Catalog, you can set the properties that display the top and bottom most frequent sample values and patterns of frequency distributions. You can change these settings in the Data Catalog configuration window.
Perform the following steps to set up the most frequent values and patterns:
Procedure
Navigate to Manage and then click Configuration.
Determine whether the configuration property should be set on the LDC Metadata Server, the LDC Agent, or the LDC Application Server, and scroll to the applicable area of the Configuration page.
Expand the MISC section.
Expand the Number of top and bottom patterns count property.
Enter an integer to be split between top and bottom patterns and click Save.
By default, this value is set to 6, which is split into 3 top and 3 bottom patterns.Return to the Configuration page and expand DISCOVERY_PROFILER in the applicable section.
Expand the Memory multiplier for most frequent values and patterns property.
Enter an integer for memory slots for most frequent values and most frequent patterns to use during profiling.
Click Save.