Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com

 

Hitachi Vantara Lumada and Pentaho Documentation

Introduction

Parent article

Plugin framework details and deployment artifacts are detailed in this section.

Framework

Plug-in terminologies are described in this section.

Basic vocabulary

Specific terms and definitions are used to describe plugins and their implementation.

Plugin

Plugins offer ways to extend the functionality of the core product and to integrate nicely with other applications. A plugin consists of the following:

  • Plugin descriptor: A JSON file describing the plugin.
  • Plugin jar: Needed only for "back-end" actions, this jar contains the implementation of business logic for the plugin functionality.

Intent

A plugin exposes one or more intents (or actions). There are two types of intents:

  • Client-side intents: Those that execute purely in the end user's browser, for example, "Open link".
  • Server-side intents: Those that execute in the application server, for example, "Send Email".

Variable

A plugin may define zero or more variables that configure the behavior of the plugin. For example, for email actions, variables will define the email server address, port, credentials, etc. For open link actions, variables can be the link address.

The plugin descriptor

This is a JSON file describing the plugin. Here's a snippet.

{  
  "name" : "Email Plugin",  
  "label" : "Email",  

  "serverSideIntents" : [  
    {...}  
  ],  
  "clientSideIntents" : [  
    {...}  
  ],  
  "variables" : [  
    {...}  
  ]  
}

  • name

    Uniquely identifies a plugin.

  • label

    Display friendly label for a plugin, may be used in the future to group multiple intents together as a menu.

  • serverSideIntents

    A list of server side action descriptors.

  • clientSideIntents

    A list of client side action descriptors

  • variables

    A list of variables declared by the plugin. These variables are available for all intents declared by this plugin

Variables

Variables are values that could be configured at deployment time, and used within the intents. For example, in the Send Email action, the mail server and authentication properties can be configured using variables. Variable definition looks like this:

{  
    ...  

    "variables": [  
        {  
            "name": "mail.smtp.host",  
            "value": "smtp.gmail.com",  
            "type": "STRING"  
        },  
        {  
            "name": "mail.smtp.port",  
            "value": 587,  
            "type": "INT"  
        },  
        {  
            "name": "mail.smtp.starttls.enable",  
            "value": true,  
            "type": "BOOLEAN"  
        },  
        {  
            "name": "cc",  
            "label": "Cc",  
            "description": "Comma separated list of recipients to send a copy to",  
            "required": true,  
            "userEditable": true  
        }  
        ...  
    ]  
    ...  
}

  • name

    An internal name for the variable. Intents use the name to retrieve a variable value.

  • value

    The default value of the variable. The value can be parameterized - for example, ${resourcePath}, ${userName}. See below for further details.

  • type

    The data-type of the variable. The allowed types are : LONG, DOUBLE, STRING, PASSWORD, BOOLEAN. Date values can be modelled as UNIX timestamp in milliseconds (LONG).

  • userEditable

    Marking variables as user-editable offers them to be changed/populated at runtime when invoking actions from UI.

  • label

    This is a user-friendly label on the variable for user-editable variables.

  • description

    This is help text that will be shown to end-users for user editable variables.

  • required

    Adds a validation for non-empty values at runtime for user editable variables.

Variables can be used to retrieve special runtime variables by using the ${xxx} pattern in the values. For example, consider the following:

{  
    "name": "myResourceId",  
    "value": "${resourceKey}",  
    "type": "STRING"  
}

At runtime, the value of the resource id in context will be substituted and will be available as the value of the variable myResourceId in the action implementation class.

These dollar $variables can be used both in server-side as well as client-side actions. For example, a client-side action may expose specify a open URL template of the form

"url_template: http://54.67.46.235/create/dataset?uri=${resourcePath}"

which for instance, might resolve to:

http://54.67.46.235/create/dataset?uri=/user/jdoe/data/file.csv

The following dollar variables are available:

  • User properties

    • ${userName}: Username of the currently logged in user
    • (could be expanded to user.email etc., in the future)
  • Resource properties

    • ${resourceKey}: Resource ID of the selected resource
    • ${resourcePath}: /user/jdoe/file.csv
    • ${resourceName}: file.csv
  • Data source properties

    (For the resource in context, the source to which the resource belongs)

    • ${sourceKey}: Source id
    • ${sourceName}: "MyHDFS"
    • ${sourcePath}: Source path for the data source, for example, /data
    • ${sourceType}: source_type_hdfs | source_type_hive | source_type_jdbc
    • ${sourceUrl}: hdfs://namenode:8020/

Server-side intent

A server-side action is one that executes in the application server (as against purely on the browser side for client-side action). It is defined in the plugin descriptor as follows:

{  
    ...  

    "serverSideIntents": [  
        "name": "send_email",  
      "label": "Email Link",  
      "handler": "com.ldc.plugins.email.EmailIntentHandler",  
      "variables": [  
        {  
          "name": "template",  
          "value": "Hello,nn${userName} sent you this link: http://neptune:8082/browse/${resourceKey}nn- Lumada Data Team.",  
          "type": "STRING"  
        },  
        {  
          "name": "to_email",  
          "label": "To",  
          "description": "Comma separated list of recipients",  
          "value": "Tushar Inamdar<tushar@ldc.com>",  
          "type": "STRING",  
          "required": true,  
          "userEditable": true  
        }  
      ],  
      "contexts": [  
        "BROWSE",  
        "SEARCH"  
      ],  
      "constraints": {  
        "entityTypes": [  
          "DATA_SOURCE",  
          "DATA_RESOURCE",  
          "TAG_DOMAIN",  
          "TAG"  
        ],  
        "resourceTypes": [  
          "HDFS_FOLDER",  
          "HDFS_FILE",  
          "HIVE_DATABASE",  
          "HIVE_TABLE",  
          "JDBC_DATABASE",  
          "JDBC_TABLE"  
        ],  
        "userRoles": [  
          "ADMINISTRATOR",  
          "BUSINESS_ANALYST"  
        ]  
      }  
    ],  

    ...  
}

  • name

    The name that identifies this action within the plugin.

  • label

    The label is a user-displayable name of the action. For example, it will be shown in the contextual menu when invoking actions on resources.

  • handler

    The java class name for the action's implementation. More on this below.

  • variables

    A list of configurable variables for this intent.

  • contexts

    The areas in the UI that should expose this action.

  • constraints

    The filters to be applied before showing the action menu. Constraints are for multiple aspects of the target entity:

    • entityTypes: Show this action only for these entity types. Valid values are: DATA_SOURCE, DATA_RESOURCE, TAG_DOMAIN, TAG, USER, ROLE.
    • resourceTypes: HDFS_FOLDER, HDFS_FILE, HIVE_DATABASE, HIVE_TABLE, JDBC_DATABASE, JDBC_TABLE.
    • userRoles: Show this action only to users in these roles:

Implementing server-side intents

Server-side actions are implemented in Java. The ldc-plugin-api should be added as a dependency, which contains public classes and API that can be used to implement custom plugin actions.

The main interface to implement is the IntentHandler:

public interface IntentHandler { 
    IntentResponse handleIntent(Intent intent);  
}

The Intent parameter contains the context data:

public interface Intent {
    Map<String, Variable> getData();
    Logger getLogger();  
}

  • The getData() method returns a map of variables. Wherever parameterized, the variables values are substituted.
  • The getLogger() method returns a slf4j-like logger interface which can and should be used to log plugin events.

public interface Logger {
    void debug(String message, Throwable throwable);
    void info(String message, Throwable throwable);
    void warn(String message, Throwable throwable);
    void error(String message, Throwable throwable);  

    void debug(String message, Object... args);
    void info(String message, Object... args);
    void warn(String message, Object... args);
    void error(String message, Object... args);
}

The handleIntent method returns an IntentResponse:

public interface IntentResponse {
    String getMessage();
    Status getStatus();

    enum Status {SUCCESS, FAILURE}  
}

The IntentResponse is used to display a custom message generated by the plugin to the end user.

For long running tasks, implementations may use the PluginUtilities.invokeAsync() method and supply a runnable.

PluginUtilities.invokeAsync (new Runnable(){
    @Override public void run(){
        try {
            performLongRunningTaskHere();
        } catch (IOException e) {
            intent.getLogger().error("", e);
        }
    }
Encrypting passwords In plugin descriptors

Passwords occurring in the plugin descriptor json can be encrypted using the standard Lumada Data Catalog encryption utility and used in variables of type PASSWORD:

"variables": [  
     {  
      "name": "password",  
      "value": "enc(rO0ABXcIAAABXIB2RwdzcgAZamF2YXguY3J5cHRvLlNlYWxlZE9iamVjdD42PabDt1RwAgAEWwANZW5jb2RlZFBhcmFtc3QAAltCWwAQZW5jcnlwdGVkQ29udGVudHEAfgABTAAJcGFyYW1zQWxndAASTGphdmEvbGFuZy9TdHJpbmc7TAAHc2VhbEFsZ3EAfgACeHBwdXIAAltCrPMX+AYIVOACAAB4cAAAACCHi3NzySa5e08Ip92BJbuAIuhJ7yH8Gy5Y5wFfwzfXPHB0ABRBRVMvRUNCL1BLQ1M1UGFAZGluZw==)",  
      "type": "PASSWORD"  
    }  
  ]

Client-side intent

A client-side intent is one that executes on the browser side. Think of actions like Open in Hue which opens up a new browser tab pointing to that resource in Hue. It is defined in the plugin descriptor as follows:

{  
    ...  

"clientSideIntents" : [  
    {  
      "name": "open_link",  
      "label": "Open in Paxata",  
      "handler": "com.hitachivantara.datacatalog.plugins.uri.OpenUri",  
      "variables": [  
        {  
          "name": "template",  
          "value": "https://trial.paxata.com/#/project/acb5260db1664946830968f46a4bcd37/ds-acb5260db1664946830968f46a4bcd37/steps",  
          "type": "STRING"  
        }  
      ],  
      "contexts": [  
        "BROWSE",  
        "SEARCH"  
      ],  
      "constraints": {  
        "entityTypes": [  
          "DATA_RESOURCE"  
        ],  
        "resourceTypes": [  
          "HDFS_FOLDER",  
          "HDFS_FILE",  
          "HIVE_DATABASE",  
          "HIVE_TABLE",  
          "JDBC_DATABASE",  
          "JDBC_TABLE"  
        ],  
        "userRoles": [  
          "ADMINISTRATOR",  
          "DATA_STEWARD"  
        ]  
      }  
    }  
  ]  
}

  • handler

    There is only one implementation of a client-side action - com.hitachivantara.datacatalog.plugins.uri.OpenUri, which is bundled. Third-parties cannot provide customized implementations of client-side actions.

How to deploy

The deliverable artifacts are:

  • A plugin descriptor file (must end in -descriptor.json).
  • A JAR file containing the implementation for the server-side actions (not required for client-side only actions).
  • Any JARs dependencies.

Put all of the above artifacts in the /ext/ directory and restart the app server for changes to take effect. During restart, pay attention to startup logs as the plugins are picked up and initialized.