Skip to main content

Pentaho+ documentation has moved!

The new product documentation portal is here. Check it out now at docs.hitachivantara.com

 

Hitachi Vantara Lumada and Pentaho Documentation

Create database plugins

Parent article

PDI uses database plugins to support specific database systems beyond generic JDBC functionality.

A database plugin helps in the following areas:

  • Constructing connection strings
  • Passing connection settings to JDBC
  • Dialect-aware SQL generation
  • Detecting special abilities and limitations of JDBC drivers

Understanding database plugins

A database plugin introduces a new entry in the PDI database dialog box.Database Connection dialog box

This section explains the architecture and programming concepts for creating your own database plugin. We recommend that you open and refer to the sample database plugin sources while following these instructions.

PDI database plugins consist of a single Java class that implements the interface org.pentaho.di.core.database.DatabaseInterface.

In order for PDI to recognize the database plugin, the class implementing DatabaseInterface must also be annotated with the Java annotation org.pentaho.di.core.plugins.DatabaseMetaPlugin.

Supply these annotation attributes.

  • type

    A globally unique ID for database plugin

  • typeDescription

    The label to use in the database dialog box

It is recommended to extend org.pentaho.di.core.database.BaseDatabaseMeta, which provides default implementations for most of the methods in DatabaseInterface. Existing PDI database interfaces are a great source of information when developing a new database plugin.

The following section classifies some of the most commonly overridden methods. They can be roughly classified into three subject areas: information about connections, SQL dialect, and general capability flags.

Connection Details

These methods are called when PDI establishes a connection to the database, or the database dialog is populated with database-specific defaults.

  • public String getDriverClass()
  • public int getDefaultDatabasePort()
  • public int[] getAccessTypeList()
  • public boolean supportsOptionsInURL()
  • public String getURL()
SQL Generation

These methods are called when PDI constructs SQL.

  • public String getFieldDefinition()
  • public String getAddColumnStatement()
  • public String getSQLColumnExists()
  • public String getSQLQueryFields()
Capability Flags

These methods are called when PDI determines the run-time characteristics of the database system. For instance, the database systems may support different notions of metadata retrieval.

  • public boolean supportsTransactions()
  • public boolean releaseSavepoint()
  • public boolean supportsPreparedStatementMetadataRetrieval()
  • public boolean supportsResultSetMetadataRetrievalOnly()

Exploring existing database implementations

PDI sources are invaluable when seeking example implementations of databases. Each of the PDI core database support classes is located in the org.pentaho.di.core.database package found in the core/src folder.

For example, here are the classes that define behavior for some major database systems.

DatabaseDatabaseInterface Class
MySQLorg.pentaho.di.core.database.MySQLDatabaseMeta
Oracleorg.pentaho.di.core.database.OracleDatabaseMeta
PostgreSQLorg.pentaho.di.core.database.PostgreSQLDatabaseMeta

When implementing a database plugin for a new database system, we recommended starting from an existing database class that already shares characteristics with the new database system.

Deploying database plugins

To deploy your database plugin, perform the following steps:

Procedure

  1. Create a JAR file containing your plugin class(es).

  2. Create a new folder, give it a meaningful name, and place your JAR file inside the folder.

  3. Place the plugin folder you just created in a specific location for PDI to find.

    Depending on how you use PDI, you need to copy the plugin folder to one or more locations as per the following list.
    • Deploying to the PDI client or Carte: Copy the plugin folder into design-tools/data-integration/plugins/databases, then restart the PDI client. After restarting the PDI client, the new database type is available from the PDI database dialog box.
    • Deploying to the Pentaho Server for Data Integration: Copy the plugin folder into server/pentaho-server/pentaho-solutions/system/kettle/plugins/databases, then restart the server. After restarting the Pentaho Server, the plugin is available to the server.
    • Deploying to Pentaho Server for Business Analytics: Copy the plugin folder into server/pentaho-server/pentaho-solutions/system/kettle/plugins/databases, then restart the server. After restarting thePentaho Server, the plugin is available to the server.
  4. When deploying database plugins, make sure to also deploy the corresponding JDBC drivers. See Specify data connections for the Pentaho Server for instructions about adding JDBC drivers.

Sample database plugin

The sample database plugin project is designed to show an implementation of a database plugin that you can use as a basis to develop your own database plugins.

The sample database plugin registers the CSV JDBC driver from http://csvjdbc.sourceforge.net/ as a database in PDI. This enables reading from CSV files in a directory using basic SQL.

The included sample transformation in demo_transform/demo_database.ktr uses the database plugin to read a basic CSV file through JDBC.

Follow these steps in order to build and deploy the sample plugin.

Procedure

  1. Obtain the sample plugin source.

    The database plugin source is available in the download package. Download the package and unzip it. The database plugin resides in the kettle-sdk-database-plugin folder.
  2. Configure the build by opening kettle-sdk-database-plugin/build/build.properties and setting the kettle-dir property to the base directory of your PDI installation.

  3. Build and deploy.

    You may choose to build and deploy the plugin from the command line, or work with the Eclipse IDE instead. Both options are described below.
  4. You can test the new plugin using the transformation from the database plugin demo_transform folder.

Results

Results of database plugin example

Build and deploy sample database plugin from the command line

The plugin is built using Apache Ant. Build and deploy the plugin from the command line by invoking the install target from the build directory.

kettle-sdk-database-plugin $ cd build
build $ ant install

The install target compiles the source, creates a JAR file, creates a plugin folder, and copies the plugin folder into the plugins/databases directory of your PDI installation. It also copies csvjdbc.jar to PDI's lib/ directory, which provides the JDBC driver the plugin depends on.

Build and deploy sample database plugin from Eclipse

Before you begin

Import the plugin sources into Eclipse:
  1. From the menu, select File Import Existing Projects Into Worksapace.
  2. Browse to the kettle-sdk-database-plugin folder and choose the project to be imported.
To build and install the plugin, follow these steps:

Procedure

  1. Open the Ant view in Eclipse by selecting Window Show View from the main menu and select Ant.

    You may have to select Other Ant if you have not used the Ant view before.
  2. Drag the file build/build.xml from your project into the Ant view, and execute the install target by double-clicking it.

  3. After the plugin has been deployed, restart the PDI client.