Configuring AEL with Spark in a secure cluster
The AEL daemon works in an unsecured cluster by default. You can secure communication channels between the PDI client and the AEL daemon server and also between the AEL daemon server and the Spark driver using SSL (Secure Sockets Layer), Kerberos, or both. If your AEL daemon server and your cluster machines are in a secure environment like a data center, you may only want to configure a secure connection between the PDI client and the AEL daemon server.
Authentication with Kerberos
To enable security, you can configure the AEL daemon to work in a secure cluster using impersonation. Kerberos authentication can be used with AEL in two ways: with the connection from the client to the AEL daemon and with the Spark submit process.
Setup a secure client connection
Download and install Kerberos server. Refer to Set Up Kerberos for Pentaho for further details on installing the Kerberos server.
Create a keytab and principal to use for your client access.
Open the PDI client and choose.
Add the properties KETTLE_AEL_PDI_DAEMON_KEYTAB and KETTLE_AEL_PDI_DAEMON_PRINCIPAL and set the values to the location of the keytab and principal, respectively.
Restart the PDI client.
Setup a secure server connection
Create a keytab and server principal to use for your server access.
Navigate to the adaptive-execution/config/application.properties file and open it with a text editor. Set the values for your environment as in the following table:
Parameter Value keytabLocation Path to the keytab used for the Kerberos principal. kerberosPrincipal Path to the Kerberos service principal that has the authority to impersonate another user. disableProxyUser The AEL daemon can impersonate a proxy user when authenticating to your secure cluster. Set to true to disable the proxy user. The acting user will then be the Kerberos service principal. The default value is false.
Using SSL encryption
Set up SSL security by following the instructions in the article Enable SSL in the Pentaho Server with a certificate authority.
Import your certificate to the Java keystore on the machine where the PDI client is installed. If the Pentaho Server is installed on a different machine, import the certificate to the Java keystore on that machine.
At the following prompts, enter a new password and enter Y:
Enter keystore password: Trust this certificate?
Configure the daemon for SSL
Navigate to the adaptive-execution/config/application.properties file and open it with a text editor.
Set the values for your environment as in the following table:
Parameter Value websocketURL The fully-qualified domain name of the node where the AEL daemon is installed. For example,
ael.ssl.enabled true ael.ssl.key-store /users/myusername/pentaho/mycertificate.p12 ael.ssl.key-store-type PKCS12 ael.ssl.key-store-password The SSL keystore password. This must be set to your keystore password. ael.ssl.key-password The SSL key password. This must be set to your key password.