Increase the DI Server Memory Limit
Overview
Explains how to increase memory limits.
We recommend that you increase PDI's memory limit so the DI Server and Data Integration Design Tool (Spoon) can perform memory-intensive tasks, like sort or group large datasets or run complex transformations and jobs. You will need to increase the memory limit for both the DI Server and Spoon. If you choose to not increase the memory limit, PDI uses the default memory settings in the PDI startup scripts.
The way that you increase the memory settings for the DI Server depends on how you installed PDI and what operating system you are running it on.
Increase DI Server Memory Limit for Wizard Installations on Windows
If you used the Installation Wizard to install the DI Server on a Windows machine, you can increase the DI Server's memory limits by editing the Java memory settings for Tomcat. (Tomcat, which is installed with the installation wizard, is the web application server that the DI Server runs on.) If you didn't use the Wizard or you are not running PDI on a Windows machine, check out the other ways to increase memory.
- Stop the Pentaho DI Server if it is running.
- Type services.msc into the Windows Search Box.
- Find the Pentaho Server name (Data Integration Server) and open it so you can find the service name. The service name should appear at the top of the first tab (General). It will be pentahoDataIntegrationServer.
- Go into the bin file (C:\Program Files\pentaho\server\data-integration-server\tomcat\bin\) and rename the tomcat6w.exe file to match the service name (pentahoDataIntegrationServerw.exe). This will ensure that the server starts with the software.
- After you have renamed the file, open it by double-clicking on it. This will not open the file, it will allow you to edit it. You may need to right-click and select Run as Administrator. This depends on your user permission settings. The Properties Window will open.
- Select the Java tab.
- Set the memory setting to a minimum of 4096 M and a maximum of 6144 M, depending on your computer's memory capabilities.
- Start the Tomcat server or service. Note: Make sure to also increase the Spoon memory limit.
Increase DI Server Memory Limit for Wizard Installations on Linux
If you used the Wizard to install PDI on your Linux machine, you can increase memory limits by editing a variable in one of the Pentaho-supplied scripts. If you didn't use the Wizard or you are not running PDI on a Linux machine, see the other ways to increase memory.
- Go to /pentaho/server/data-integration-server/tomcat/bin/ directory and run the ./shutdown.sh command to stop the appropriate server.
- Change the directory to data-integration-server/scripts.
- Edit the ctl.sh file.
- Locate the line under start tomcat, which looks like this:
export JAVA OPTS="-Dpentaho.installed.licenses.file=/opt/pentaho/.installedLicenses.xml -Xms128m Xmx768m -XX-MaxPermSize=256m -Dsun.rmi.dyc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000"
- Set the memory to a minimum of 4096 M and a maximum of 6144 M, depending on your computer's memory capabilities.
- Start the Tomcat server or service. Note: Make sure to also increase the Spoon memory limit.
Increase DI Server Memory Limit for Custom Installations on Windows or Linux
If you used a custom method to install PDI on your Windows or Linux machines, you can increase memory limits by editing a text file. If you used the Wizard to install PDI, then check out the other the other ways to increase memory.
- Windows:
- Install with Your Own DI Repository: Modify Tomcat Startup Script
- DI Server Manual Installation: Modify Tomcat and JBoss Startup Scripts
- Linux:
- Install with Your Own DI Repository: Modify Tomcat Startup Script
- DI Server Manual Installation: Modify Tomcat and JBoss Startup Scripts