Upgrading the JDK

Cloudera Manager and CDH require a supported version of the Java Development Kit (JDK) to be installed on all hosts. For details, see Java Requirements

Supported JDKs
Cloudera Enterprise Version Supported JDK
5.3 -5.15 Oracle JDK 1.7, Oracle JDK 1.8
5.16 and higher 5.x releases Oracle JDK 1.7, Oracle JDK 1.8, OpenJDK 1.8
6.0 Oracle JDK 1.8
6.1 Oracle JDK 1.8, OpenJDK 1.8
6.2 Oracle JDK 1.8, OpenJDK 1.8

.

Loading Filters ... 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0

There are two procedures you can use to upgrade the Oracle JDK:
  • Installing JDK 8 as part of an Upgrade to Cloudera Manager 6.x

    If you are upgrading to Cloudera Manager 6.0.0 or higher, you can manually install JDK 1.8 on the Cloudera Manager server host, and then, as part of the Cloudera Manager upgrade process, you can specify that Cloudera Manager upgrade the JDK on the remaining hosts.

  • Manually Installing Oracle JDK 1.8

    You can manually install JDK 1.8 on all managed hosts. If you are upgrading to any version of Cloudera Manager 5.x, you must use this procedure. Continue with the steps in the next section.

To upgrade to OpenJDK, see Manually Migrating from Oracle JDK to OpenJDK

Manually Installing Oracle JDK 1.8

You can manually install Oracle JDK 1.8 on all managed hosts. If you are upgrading to any version of Cloudera Manager 5.x, you must use the following procedure:

  1. Download the .tar.gz file for one of the 64-bit versions of Oracle JDK 1.8 from Java SE 8 Downloads. (This link is correct at the time of writing, but can change.) See Supported Java versions for CDH 5 or Java Requirements.
  2. Perform the following steps on all hosts that you are upgrading:
    1. Log in to the host as root using ssh.
    2. Copy the downloaded .tar.gz file to the host.
    3. Extract the JDK to the folder /usr/java/jdk-version. For example:
      tar xvfz /path/to/jdk-8u<update_version>-linux-x64.tar.gz -C /usr/java/
  3. If you have configured TLS for Cloudera Manager, copy the jssecacerts file from the previous JDK installation to the new JDK installation. This step is not required when using JDK 1.8.0_162 or greater. JDK 1.8.0_162 enables unlimited strength encryption by default.
    For example:
    cp previous_java_home/jre/lib/security/jssecacerts new_java_home/jre/lib/security
    (Substitute previous_java_home and new_java_home with the paths to the JDK installations.)
  4. Configure the location of the JDK on cluster hosts.
    1. Open the Cloudera Manager Admin Console.
    2. In the main navigation bar, click the Hosts tab. If you are configuring the JDK location on a specific host only, click the link for that host.
    3. Click the Configuration tab.
    4. Select Category > Advanced.
    5. Set the Java Home Directory property to the custom location.
    6. Click Save Changes.
  5. On the Cloudera Manager Server host only (not required for other hosts):
    1. Open the file /etc/default/cloudera-scm-server in a text editor.
    2. Edit the line that begins with export JAVA_HOME (if this line does not exist, add it) and change the path to the path of the new JDK (you can find the path under /usr/java).
      For example: (RHEL and SLES)
      export JAVA_HOME="/usr/java/jdk1.8.0_141-cloudera"
      For example: (Ubuntu)
      export JAVA_HOME="/usr/lib/jvm/java-8-oracle-cloudera"
    3. Save the file.
    4. Restart the Cloudera Manager Server.
      RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
      sudo systemctl restart cloudera-scm-server
      RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
      sudo service cloudera-scm-server restart
  6. Restart the Cloudera Management Service.
    1. Log in to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Restart.
  7. Restart all clusters:
    1. On the Home > Status tab, click to the right of the cluster name and select Restart.
    2. Click Restart that appears in the next screen to confirm. If you have enabled high availability for HDFS, you can choose Rolling Restart instead to minimize cluster downtime. The Command Details window shows the progress of stopping services.

      When All services successfully started appears, the task is complete and you can close the Command Details window.

  8. Delete the files from your previous Java installation. If you do not delete these files, Cloudera Manager and other components may continue to use the old version of the JDK.

OpenJDK

You must install a supported version of OpenJDK. If your deployment uses a version of OpenJDK lower than 1.8.0_181, see TLS Protocol Error with OpenJDK.

Manually Installing OpenJDK

Before installing Cloudera Manager and CDH, perform the steps in this section to install OpenJDK on all hosts in your cluster(s).

When you install Cloudera Enterprise, Cloudera Manager includes an option to install Oracle JDK. De-select this option.

See Supported JDKs for information on which JDK versions are supported for Cloudera Enterprise releases.

Log in to each host and run the following command:
RHEL
su -c yum install java-1.8.0-openjdk-devel
Ubuntu
sudo apt-get install openjdk-8-jdk
SLES
sudo zypper install java-1_8_0-openjdk-devel

Manually Migrating from Oracle JDK to OpenJDK

If you have Oracle JDK installed on the hosts managed by Cloudera Manager, use the steps in this section to migrate your deployment to use OpenJDK. The steps below require you to restart all clusters, which will cause downtime as the hosts restart. If your clusters have enabled high availability for HDFS, you can use a rolling restart to restart the clusters without downtime. Note that until the rolling restart completes, some of the hosts in your cluster will still be using the Oracle JDK. If you do not want a temporarily mixed environment, you can stop the cluster before performing the steps in this section to migrate the JDK.

You must install a supported version of OpenJDK. If your deployment uses a version of OpenJDK lower than 1.8.0_181, see TLS Protocol Error with OpenJDK.

  1. Find out the package name of your currently installed Oracle JDK by running the following commands. The grep commands attempt to locate the installed JDK. If the JDK package is not returned, try looking for the string sdk.
    RHEL
    yum list installed |grep oracle
    Ubuntu
    apt list --installed | grep oracle
    SLES
    zypper search --installed-only |grep oracle
    The command will return values similar to the following example::
    oracle-j2sdk1.7.x86_64              1.7.0+update67-1           @cloudera-manager
    The Oracle JDK package name in the above example is: oracle-j2sdk1.7.x86_64.
  2. Locate the version of Java that is installed: Open the Cloudera Manager Admin Console and go to Support > About. The Java version displays along with other version information.
  3. Log in to each host managed by Cloudera Manager (including the Cloudera Manager server host) and run the following command to install OpenJDK:
    RHEL
    su -c yum install java-1.8.0-openjdk-devel
    Ubuntu
    sudo apt-get install openjdk-8-jdk
    SLES
    sudo zypper install java-1_8_0-openjdk-devel
  4. On the Cloudera Manager Server host only (not required for other hosts):
    1. Open the file /etc/default/cloudera-scm-server in a text editor.
    2. Edit the line that begins with export JAVA_HOME (if this line does not exist, add it) and change the path to the path of the new JDK (the JDK is usually installed in /usr/lib/jvm)(or /usr/lib64/jvm on SLES 12), but the path may differ depending on how the JDK was installed).

      For example:

      RHEL 6
      export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk.x86_64"
      RHEL 7
      export JAVA_HOME="/usr/lib/jvm/java-1.8.0-openjdk"
      Debian
      export JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64"
      Ubuntu
      export JAVA_HOME="/usr/lib/jvm/openjdk-8-jdk"
      SLES
      export JAVA_HOME="/usr/lib64/jvm/java-1.8.0-openjdk"
    3. Save the file.
    4. Restart the Cloudera Manager Server.
      RHEL 7, SLES 12, Debian 8, Ubuntu 16.04 and higher
      sudo systemctl restart cloudera-scm-server
      RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04 or 14.04
      sudo service cloudera-scm-server restart
  5. Restart the Cloudera Management Service.
    1. Log in to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Restart.
  6. Restart all clusters:
    1. On the Home > Status tab, click to the right of the cluster name and select either Restart or Rolling Restart. Selecting Rolling Restart minimizes cluster downtime and is available only If you have enabled high availability for HDFS.
    2. Click Restart or Rolling Restart that appears in the next screen to confirm. The Command Details window shows the progress of stopping services.

      When All services successfully started appears, the task is complete and you can close the Command Details window.

  7. Remove the Oracle JDK:
    1. Run the following command using the Java version to verify the current running version of Java:
      /usr/java/jdk<Java version string>-cloudera/bin/java -version
    2. Perform the following steps on all hosts managed by Cloudera Manager:
      1. Run the following command to remove the Oracle JDK: (If you do not delete these files, Cloudera Manager and other components may continue to use the old version of the JDK.)
        RHEL
        yum remove oracle-<JDK package name>
        Ubuntu
        apt-get remove oracle-<JDK package name>
        SLES
        zypper rm oracle-<JDK package name>
      2. Confirm that the package has been removed:
        RHEL
        yum list installed |grep -i oracle
        Ubuntu
        apt list --installed | grep -i oracle
        SLES
        zypper search --installed-only |grep -i oracle

Using AES-256 Encryption

If you are using CentOS/Red Hat Enterprise Linux 5.6 or higher, or Ubuntu, which use AES-256 encryption by default for tickets, you must install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy File on all cluster and Hadoop user machines. For JCE Policy File installation instructions, see the README.txt file included in the jce_policy-x.zip file. For more information, see Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy File

Alternately, you can configure Kerberos to not use AES-256 by removing aes256-cts:normal from the supported_enctypes field of the kdc.conf or krb5.conf file. After changing the kdc.conf file, you must restart both the KDC and the kadmin server for those changes to take affect. You may also need to re-create or change the password of the relevant principals, including, potentially the Ticket Granting Ticket principal (krbtgt/REALM@REALM). If AES-256 is still used after completing steps, the aes256-cts:normal setting existed when the Kerberos database was created. To fix this, create a new Kerberos database and then restart both the KDC and the kadmin server.

To verify the type of encryption used in your cluster:

  1. On the local KDC host, type this command to create a test principal:
    kadmin -q "addprinc test" 
  2. On a cluster host, type this command to start a Kerberos session as the test principal:
    kinit test 
  3. On a cluster host, type this command to view the encryption type in use:
    klist -e 

    If AES is being used, output like the following is displayed after you type the klist command; note that AES-256 is included in the output:

    Ticket cache: FILE:/tmp/krb5cc_0
    Default principal: test@SCM
    Valid starting     Expires            Service principal
    05/19/11 13:25:04  05/20/11 13:25:04  krbtgt/SCM@SCM
        Etype (skey, tkt): AES-256 CTS mode with 96-bit SHA-1 HMAC, AES-256 CTS mode with 96-bit SHA-1 HMAC 

Configuring a Custom Java Home Location

Although not recommended, the Java Development Kit (JDK), which Cloudera services require, may be installed at a custom location if necessary. These steps assume you have already installed the JDK as documented in Step 2: Install Java Development Kit or as part of an upgrade.

To modify the Cloudera Manager configuration to ensure the JDK can be found:
  1. Open the Cloudera Manager Admin Console.
  2. In the main navigation bar, click the Hosts tab. If you are configuring the JDK location on a specific host only, click the link for that host.
  3. Click the Configuration tab.
  4. Select Category > Advanced.
  5. Set the Java Home Directory property to the custom location.
  6. Click Save Changes.
  7. Restart all services.