These instructions provide the steps necessary to deploy Docker images of the Carte Server on GCP.
Prerequisites for installing the Carte Server on GCP
The following software must be installed on your workstation before installing the Carte Server:
- The PDI is needed to connect to the Carte Server for testing.
- A stable version of Docker must be installed on your workstation. See Docker documentation.
- The Kubernetes command-line tool, Kubectl must be installed.
- (Optional) Use Kubernetes or Lens to manage your Kubernetes cluster.
- (Optional) Use the Kubectl bash-completion package.
- GCloud CLI utils must be installed and authenticated.
Process overview for running the Carte Server on GCP
Use the following steps to deploy the Carte Server on the GCP cloud platform
- Download and extract the Carte Server for GCP.
- Create a Docker registry in GCP.
- Push the Carte Server Docker image to GCP.
- Create and populate a Google Cloud Storage bucket.
- Create a Google Cloud SQL PostgreSQL instance.
- Set up a GKE cluster on GCP.
- Deploy the Carte Server on GCP.
Download and extract Pentaho for GCP
Download and open the package files that contain the files needed to install Pentaho.
Navigate to the Support Portal and download the GCP version of the Docker image with the corresponding license file for the applications you want to install on your workstation.NoteMake note of the image name for later.
Extract the image into your local Docker registry.The image package file (<package-name>.tar.gz) contains the following:
Name Content description image Directory containing all the Pentaho source images. yaml Directory containing YAML configuration files and various utility files. README.md File containing a link to detailed information about what we are providing for this release.
Create a Docker registry in GCP
Before pushing the Pentaho image to GCP, you need to create a Docker registry in GCP.
Create a Docker registry in GCP.For instructions, see Store Docker container images in Artifact Registry.
Connect to the Docker registry using the following command:
gcloud auth configure-docker <YOUR_REGION>-docker.pkg.dev
To verify that the registry has been added correctly, run this command:
Record the name of the registry that you have created in the Worksheet for GCP hyperscaler.
Load and push the Pentaho Docker image to the GCP registry
Perform the following steps to load and push the Pentaho Docker image to GCP:
Navigate to the image directory containing the Pentaho tar.gz files.
Select and load the tar.gz file into the local registry by running the following command:
docker load -i <pentaho-image>.tar.gz
Record the name of the source image that was loaded into the registry by using the following command:
Tag the source image so it can be pushed to the cloud platform by using the following command:
docker tag <source-image>:<tag> <target-repository>:<tag>
Push the image to the GCP registry using the following command:
docker push <IMAGE_NAME>
Verfiy that the image has been properly loaded using the Google Cloud Console.
Create a Google Cloud Storage bucket
Create a Google Cloud Storage bucket and place your configuration files in any directory path in the bucket.
In these instructions, the following path is used as an example:
Perform the following steps to create and populate a Google Cloud Storage bucket:
Create a Cloud Storage bucket as explained in the GCP documentation.
Add the Kettle transformation (
KTR) and job (
KJB) files that you want to use to the bucket.
If any of your jobs or transformations use VFS connections to the Google Storage buckets, perform the following steps:
Upload a copy of your GCS credentials file to the Google Storage bucket.For example gs://pentaho-project/my-bucket/<credentials-file>.json
Update any VFS connections that use this credentials file to point to the following path: /home/pentaho/data-integration/data/<credentials-file>.json
Copy your local .pentaho/metastore folder to the Google Storage bucket.The .pentaho/ folder is located in the user home directory by default.NoteYou must edit your GCS VFS connections before copying the .pentaho/ folder. If you need to change the VFS connections, upload the GCS credentials file and update any associated GFS connections again.
Copy any license files (*.lic) needed for the product(s) you will be using to the location specified by PROJECT_GCP_LOCATION.
Set up a GKE cluster on GCP
Use the following steps to configure a GKE cluster on Google Cloud Platform (GCP).
Create a GKE cluster on GCP.Configure the cluster to meet your requirements. For a simple example, see Create a GKE cluster.
Authenticate with the cluster using the following command:
gcloud container clusters get-credentials <CLUSTER_NAME> --region=<YOUR_REGION>
Check the connection using the following command:
(Optional) Use a tool like Kubernetes or Lens to check that they are connected.
Deploy the Carte Server on GCP
Prepare to launch your service by performing some setup steps and then deploy.
Navigate to the file
carte-gcp-gke.yamlin your distribution and open it in a text editor.
Configure the YAML file as follows:
Variable Replace with IMAGE_URI The name of the image that has been loaded into the GCP Docker registry. PROJECT_CLOUD_STORAGE_LOCATION The path to the bucket. Using the sample bucket name above, this would be pentaho-project/my-bucket. METASTORE_LOCATION
The relative path to the parent folder of the .pentaho/folder.
In the example above (with the metastore inside pentaho-project/my-bucket/.pentaho/), the value is simply “.” to indicate that the parent folder of .pentaho/ is the same as the root of the bucket.
If using a custom carte-config.xml, replace the value under CARTE_CONFIG_FILE with the relative path to the custom config.
If not using a custom carte-config.xml, comment out or remove this section from the YAML file.
If the example YAML is intended to run the pod in a different namespace than carte, replace the name carte in the namespace section at the top with the desired name.In addition, replace all instances of the namespace carte with the new name.NoteThe namespace carte is used in the rest of this procedure.
Save the YAML file.
Once all the configuration changes have been made, deploy it to the cluster:
kubectl apply -f carte-gcp-gke
Verify that the service is running by executing the following command:
kubectl get pods --namespace carteAfter a few moments, you should see a pod come up. (It can take a minute to pull the image the first time you start.)
Connect to the Carte Server from the Spoon client and try running a KTR or KJB.The default port number is 8081 but can be different if the networking setup requires it.