Skip to main content
Version Pentaho
Hitachi Vantara Lumada and Pentaho Documentation

Creating Database Plugins

  1. Last updated
  2. Save as PDF

PDI uses database plugins to support specific database systems beyond generic JDBC functionality. A database plugin helps in the following areas:

  • Constructing connection strings
  • Passing connection settings to JDBC
  • Dialect-aware SQL generation
  • Detecting special abilities and limitations of JDBC drivers

A database plugin introduces a new entry in the PDI database dialog. 


database_connection3.png

This section explains the architecture and programming concepts for creating your own database plugin. We recommended that you open and refer to the sample database plugin sources while following these instructions.

Java Interface org.pentaho.di.core.database.DatabaseInterface
Base class org.pentaho.di.core.database.BaseDatabaseMeta

PDI database plugins consist of a single Java class that implements the interface org.pentaho.di.core.database.DatabaseInterface. 

In order for PDI to recognize the database plugin, the class implementing DatabaseInterface must also be annotated with the Java annotation org.pentaho.di.core.plugins.DatabaseMetaPlugin. 

Supply these annotation attributes.

Attribute Description
type A globally unique ID for database plugin
typeDescription The label to use in the database dialog

It is recommended to extend org.pentaho.di.core.database.BaseDatabaseMeta, which provides default implementations for most of the methods in DatabaseInterface. Existing PDI database interfaces are a great source of information when developing a new database plugin.

The following section classifies some of the most commonly overridden methods. They can be roughly classified into three subject areas: information about connections, SQL dialect, and general capability flags. 

1. Connection Details

These methods are called when PDI establishes a connection to the database, or the database dialog is populated with database-specific defaults.

  • public String getDriverClass()
  • public int getDefaultDatabasePort()
  • public int[] getAccessTypeList()
  • public boolean supportsOptionsInURL()
  • public String getURL()

2. SQL Generation

These methods are called when PDI constructs SQL.

  • public String getFieldDefinition()
  • public String getAddColumnStatement()
  • public String getSQLColumnExists()
  • public String getSQLQueryFields() 

3. Capability Flags

These methods are called when PDI determines the run-time characteristics of the database system. For instance, the database systems may support different notions of metadata retrieval.

  • public boolean supportsTransactions()
  • public boolean releaseSavepoint()
  • public boolean supportsPreparedStatementMetadataRetrieval()
  • public boolean supportsResultSetMetadataRetrievalOnly()

Exploring Existing Database Implementations

PDI sources are invaluable when seeking example implementations of databases. Each of the PDI core database support classes is located in the org.pentaho.di.core.database package found in the core/src folder.

For example, here are the classes that define behavior for some major database systems.

Database DatabaseInterface Class
MySQL org.pentaho.di.core.database.MySQLDatabaseMeta
Oracle org.pentaho.di.core.database.OracleDatabaseMeta
PostgreSQL org.pentaho.di.core.database.PostgreSQLDatabaseMeta

When implementing a database plugin for a new database system, we recommended starting from an existing database class that already shares characteristics with the new database system. 

Deploying Database Plugins

To deploy your plugin, follow the following steps.

  1. Create a jar file containing your plugin class(es)
  2. Create a new folder, give it a meaningful name, and place your jar file inside the folder
  3. Place the plugin folder you just created in a specific location for PDI to find. Depending on how you use PDI, you need to copy the plugin folder to one or more locations as per the following list.

    • Deploying to Spoon or Carte:

      1. Copy the plugin folder into this location: design-tools/data-integration/plugins/databases

      2. Restart Spoon. After restarting Spoon, the new database type is available from the PDI database dialog.

    • Deploying to Pentaho Server for Data Integration:

      1. Copy the plugin folder to this location: server/pentaho-server/pentaho-solutions/system/kettle/plugins/databases

      2. Restart the server. After restarting the Pentaho Server, the plugin is available to the server.

    • Deploying to Pentaho Server for Business Analytics:

      1. Copy the plugin folder to this location: server/pentaho-server/pentaho-solutions/system/kettle/plugins/databases

      2. Restart the server. After restarting the Pentaho Server, the plugin is available to the server.

  4. When deploying database plugins, make sure to also deploy the corresponding JDBC drivers. See Specify Data Connections for the Pentaho Server for instructions about adding JDBC drivers.

Sample Database Plugin

The sample database plugin project is designed to show an implementation of a database plugin that you can use as a basis to develop your own database plugins.

The sample database plugin registers the CSV JDBC driver from http://csvjdbc.sourceforge.net/ as a database in PDI. This enables reading from CSV files in a directory using basic SQL.

The included sample transformation in demo_transform/demo_database.ktr uses the database plugin to read a basic CSV file through JDBC.

Follow these steps in order to build and deploy the sample plugin.

  1. Obtain the Sample Plugin Source

    The database plugin source is available in the download package. Download the package and unzip it. The database plugin resides in the kettle-sdk-database-plugin folder.

  2. Configure the Build

    Open kettle-sdk-database-plugin/build/build.properties and set the kettle-dir property to the base directory of your PDI installation.

  3. Build and Deploy

    You may choose to build and deploy the plugin from the command line, or work with the Eclipse IDE instead. Both options are described below.

    • Build and Deploy From the Command Line

      The plugin is built using Apache Ant. Build and deploy the plugin from the command line by invoking the install target from the build directory. 

      kettle-sdk-database-plugin $ cd build
build $ ant install

      The install target compiles the source, creates a jar file, creates a plugin folder, and copies the plugin folder into the plugins/databases directory of your PDI installation. It also copies csvjdbc.jar to PDI's lib/ directory, which provides the JDBC driver the plugin depends on.

    • Build and Deploy From Eclipse

      Import the plugin sources into Eclipse:

      1. From the menu, select File > Import > Existing Projects Into Worksapace.
      2. Browse to the kettle-sdk-database-plugin folder and choose the project to be imported. 
      To build and install the plugin, follow these steps:
      1. Open the Ant view in Eclipse by selecting Window > Show View from the main menu and select Ant.

        You may have to select Other > Ant if you have not used the Ant view before.

      2. Drag the file build/build.xml from your project into the Ant view, and execute the install target by double-clicking it.
      3. After the plugin has been deployed, restart Spoon.
  4. You can test the new plugin using the transformation from the database plugin demo_transform folder.