Apply Best Practices for Kettle Logging

Kettle logging provides extensive flexibility that allows you to determine log locations, granularity, as well as what information is captured.  Here are a best practices for setting up logging in your environment.  For more information on logging, see the Logging and Monitoring Operations article.  

• Store logs in a centralized database.  By default, log files are stored locally.  Spoon, Carte, and Pentaho Server logs are stored separately.  To make log information easier to find, place logs in a central database.  As an added bonus, centralized logging makes it easier to use PDI’s performance monitoring more effectively.
• Obtain Full Insert Accesses for Tables.  Logging can fail if you do not have the appropriate accesses.  Having the appropriate accesses minimizes this risk.  To learn more about table access, consult the documentation for your database.
• Install JDBC Drivers Locally and on Each Server.  This helps you avoid Driver not found errors.  A list of supported drivers appears here.
• Use Implied Schemas When Possible.  Implied schemas result in fewer places to troubleshoot should logging fail.  Of course, you can still specify a schema if needed.
• Make templates for transformation and job files.  Include logging configurations in the template so that they can be reused with ease.
• Use Kettle Global Logging Variables When Possible.  To avoid the work of adding logging variables to each transformation or job, consider using global logging variables instead.  You can override logging variables by adding information to individual transformations or jobs as needed.

If you choose to use the kettle.properties file, observe the following best practices.

• Backup your kettle.properties files.  If you are making many changes to the kettle.properties file, consider backing up the file first.  This will make it easier to restore should issues occur.
• Maintain a Master Copy of the kettle.properties file.  It is usually easiest to use the kettle.properties file for Spoon, then to overwrite the Carte and Pentaho Server copies if changes are made.  Make sure that values that might change, such as directory paths, are maintained however.
• Test Thoroughly.  Test your settings by saving your kettle.properties file locally, then restarting spoon.  Make sure the kettle.properties file loads properly.  Execute a transformation or job that uses the settings.  Try executing the transformation locally and remotely on the Pentaho Server to ensure that log variables reference the same places.