Big Data issues
Follow the suggestions in these topics to help resolve common issues when working with Big Data:
- General configuration problems
- Cannot access cluster with Kerberos enabled
- Cannot access a hive cluster
- Cannot use Keytab file to authenticate access to PMR cluster
- HBase Get Master Failed error
- Sqoop Import into Hive fails
- Pig Job not executing after Kerberos authentication fails
- Cannot start any Pentaho components after setting MapR as active Hadoop configuration
- Kettle cluster on YARN will not start
- The Group By step is not supported in a single threaded transformation engine
- Hadoop on Windows
- Spark issues
See Pentaho Troubleshooting articles for additional topics.
General configuration problems
The issues in this section explain how to resolve common configuration problems.
Shim and configuration issues
Symptoms | Common Causes | Common Resolutions |
No shim |
|
|
Shim does not load |
|
|
The file system's URL does not match the URL in the configuration file. |
|
|
Sqoop Unsupported major.minor version Error | In Pentaho 6.0, the Java version on your cluster is older than the Java version that Pentaho uses. |
|
Connection problems
Symptoms | Common Causes | Common Resolutions |
Hostname does not resolve |
|
|
Port number does not resolve |
|
|
Cannot connect to the cluster |
|
|
Windows failure message: “java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset |
| Please follow the instructions at https://wiki.apache.org/hadoop/WindowsProblems and set the environment variable %HADOOP_HOME% to point to the directory path containing WINUTILS.EXE |
Cannot access a Hive database (Secured Clusters Only) |
|
NoteThe principal typically has a mapr prefix before the
name, for example,
mapr/mapr31.pentaho@mydomain . |
Directory access or permissions issues
Symptoms | Common Causes | Common Resolutions |
Cannot access directory |
|
|
Cannot create, read, update, or delete files or directories |
|
|
Test file cannot be overwritten |
|
|
Oozie issues
Symptoms | Common Causes | Common Resolutions |
Cannot connect to Oozie |
|
|
Zookeeper problems
Symptoms | Common Causes | Common Resolutions |
Cannot connect to Zookeeper |
|
|
Zookeeper hostname or port not found or does not resolve properly |
|
|
Kafka problems
Symptoms | Common Causes | Common Resolutions |
Cannnot connect to Kafka |
|
|
Cannot access cluster with Kerberos enabled
If this issue persists, verify the username, password, UID, and GID for each impersonated or spoofed user is the same on each node. When a user is deleted and recreated, it may then have different UIDs and GIDs causing this issue.
Cannot access a Hive cluster
If you cannot use Kerberos impersonation to authenticate and access a Hive cluster, review the steps in Use Kerberos with MapR
If this issue persists, copy the hive-site.xml file on the Hive server to the MapR distribution in these directories:
- Pentaho Server:
pentaho-server/pentaho-solutions/system/kettle/plugins/pentaho-big-data-plugin/hadoop-configurations/[mapr distribution]
- PDI client:
data-integration/plugins/pentaho-big-data-plugin/hadoop-configurations/[mapr distribution]
If this still does not work, disable pooled connections for Hive.
Cannot use keytab file to authenticate access to PMR cluster
HBase Get Master Failed error
- Pentaho Server:
pentaho-server/pentaho-solutions/system/kettle/plugins/pentaho-big-data-plugin/hadoop-configurations/[mapr distribution]
- PDI client:
data-integration/plugins/pentaho-big-data-plugin/hadoop-configurations/[mapr distribution]
Sqoop import into Hive fails
Verify the Hadoop connection information used by the local Hive installation is configured the same as the Sqoop job entry.
Cannot start any Pentaho components after setting MapR as active Hadoop configuration
As you review the instructions for configuring MapR, make sure that you have copied the required JAR files to the pentaho-big-data-plugin/hadoop-configurations/mapr3x folders for each component listed. For information on how to configure MapR, see the following references:
Pig job not executing after Kerberos authentication fails
For authentication with Pig, Pentaho uses the UserGroupInformation
wrapper around a JAAS Subject with username and password which is used for
impersonation. The UserGroupInformation
wrapper is stored in the
KMSClientProvider constructor. When the Kerberos ticket
expires, a new UserGroupInformation
is created, but the instance
stored in the KMSClientProvider constructor does not update.
The Pig job fails when Pig cannot obtain delegation tokens to authenticate the job
at execution time.
To resolve, set the key.provider.cache.expiry time to a value equal to or less than the duration time of the Kerberos ticket. By default, the key.provider.cache.expiry time is set to a value of: 10 days
Procedure
Navigate to the hdfs-site.xml file location.
- In the PDI client, navigate to: data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\hdp25
- For the Pentaho Server, navigate to: pentaho-server\pentaho-solutions\system\kettle\plugins\pentaho-big-data-plugin\hadoop-configurations\hdp25
Open the hdfs-site.xml file in a text editor.
Adjust the key.provider.cache.expiry value (in milliseconds) so that it is less than the duration time of the Kerberos ticket.
NoteYou can view the Kerberos ticket duration time in the krb5.conf file.<property> <name>dfs.client.key.provider.cache.expiry</name> <value>410000</value> </property>
Group By step is not supported in a single threaded transformation engine
- An entire set of rows sharing the same grouping key are filtered from the transformation before the Group By step.
- The Reduce single threaded option in the Pentaho MapReduce entry's Reducer tab is selected.
To fix this issue, open the Pentaho MapReduce entry and deselect the Reduce single threaded option in the Reducer tab.
Kettle cluster on YARN will not start
Verify in the File System Path (in the Files tab) that the Default FS setting matches the configured hostname for the HDFS Name node, then try starting the kettle cluster again.
Hadoop on Windows
Spark issues
Follow the suggestions in these topics to help resolve common issues with running transformations with Spark.
Steps cannot run in parallel
Some steps cannot run in parallel (on multiple nodes in a cluster), and will produce unexpected results. However, these steps can run as a coalesced dataset on a single node in a cluster. To enable a step to run as a coalesced dataset, add the step ID as a property value in the configuration file for using the Spark engine.
Get the step ID
Each PDI step has a step ID, a globally unique identifier of the step. Use either of the following two methods to get the ID of a step:
Method 1: Retrieve the ID from the log
Procedure
In the PDI client, create a new transformation and add the step to the transformation. For example, if you needed to know the ID for the Select values step, you would add that step to the new transformation.
Set the log level to debug.
Execute the transformation using the Spark engine.
The step ID will display in the Logging tab of the Execution Results pane. For example, the log will display:Selected the SelectValues step to run in parallel as a GenericSparkOperation,
where SelectValues is the step ID.
Method 2: Retrieve the ID from the PDI plugin registry
If you have created your own PDI transformation step plugin, the step ID is one of the annotation attributes that the developer supplies.
Add the step ID to the configuration file
Perform the following steps to add another step ID to the configuration file:
Procedure
Navigate to the data-integration/system/karaf/etc folder and open the org.pentaho.pdi.engine.spark.cfg file.
Append your step ID to the forceCoalesceSteps property value list, using a pipe character separator between the step IDs.
Save and close the file.
Table Input step fails
If you run a transform using the Table Input step with a large database, the step will not complete. Use one of the following methods to resolve the issue:
Method 1: Load the data to HDFS before running the transform
Run a different transformation using the Pentaho engine to move the data to the HDFS cluster.
Then use HDFS Input to run the transformation using the Spark engine.
Method 2: Increase the driver side memory configuration
Navigate to the data-integration/adaptive-execution/config folder and open the application.properties file.
Increase the value of the sparkDriverMemory parameter, then save and close the file.
User ID below minimum allowed
To resolve, change the ID of the proxy user to be higher than the minimum user ID specified for the cluster