This is the documentation for Cloudera 5.3.x.
Documentation for other versions is available at Cloudera Documentation.

Performing a Rolling Upgrade on a CDH 5 Cluster

Required Role:

  Important: This feature is available only with a Cloudera Enterprise license.
For other licenses, the following applies:
  • Cloudera Express - The feature is not available.
  • Cloudera Enterprise Data Hub Edition Trial - The feature is available until you end the trial or the trial license expires.
To obtain a license for Cloudera Enterprise, please fill in this form or call 866-843-7207. After you install a Cloudera Enterprise license, the feature will be available.

The rolling upgrade feature takes advantage of parcels and the HDFS high availability to enable you to upgrade your cluster software and restart the upgraded services without taking the entire cluster down. You must have HDFS high availability enabled to perform a rolling upgrade.

This page described how to perform a rolling upgrade between maintenance and minor versions of CDH 5, except Beta versions. For rolling upgrade between CDH 4 versions, see Performing a Rolling Upgrade on a CDH 4 Cluster.

It is not possible to perform a rolling upgrade from CDH 4 to CDH 5 because of incompatibilities between the two major versions. Instead, follow the instructions for a full upgrade at Upgrading from CDH 4 to CDH 5 Parcels.

To avoid lots of alerts during the upgrade process, you can enable maintenance mode on your cluster before you start the upgrade. This will stop email alerts and SNMP traps from being sent, but will not stop checks and configuration validations from being made. Be sure to exit maintenance mode when you have finished the upgrade in order to re-enable Cloudera Manager alerts.

The steps to perform a rolling upgrade of a cluster are as follows:

  1. Ensure High Availability is Enabled
  2. Run the Upgrade Wizard
  3. Remove CDH 5 Packages
  4. Finalize HDFS Rolling Upgrade (CDH 5.2 and higher)

Ensure High Availability is Enabled

To enable high availability, see HDFS High Availability for instructions. You do not need to enable automatic failover for rolling restart to work, though you can enable it if you wish. Automatic failover does not affect the rolling restart operation. If you have JobTracker high availability configured, Cloudera Manager will fail over the JobTracker during the rolling restart, but this is not a requirement for performing a rolling upgrade.

Run the Upgrade Wizard

The process varies depending on whether you are performing a maintenance upgrade (CDH a.b.x to CDH a.b.y. For example, CDH 5.2.0 to 5.2.1) or a minor upgrade (CDH 5.x to CDH 5.y. For example, CDH 5.2 to 5.3.).
  • Maintenance upgrade
    1. Log into the Cloudera Manager Admin console.
    2. From the Home tab Status page, click next to the cluster name and select Upgrade Cluster. The Upgrade Wizard starts.
    3. If the option to pick between packages and parcels displays, click the Use Parcels radio button.
    4. In the Choose CDH Version (Parcels) field, select the CDH version. If there are no qualifying parcels, click the click here link to go to the Parcel Configuration Settings page where you can add the locations of parcel repositories. Click Continue.
    5. Read the notices for steps you must complete before upgrading, click the Yes, I ... checkboxes after completing the steps, and click Continue.
    6. Cloudera Manager checks that hosts have the correct software installed. Click Continue.
    7. The selected parcels are downloaded and distributed. Click Continue.
    8. The wizard displays the Perform rolling upgrade to the cluster screen. For maintenance upgrades, if you check the Don't restart the cluster during this maintenance upgrade checkbox, the wizard returns to the Home tab, and you must complete the upgrade steps yourself.
    9. Leaving the Rolling Upgrade checkbox checked indicates that Cloudera Manager should upgrade services and perform a rolling restart. Click Advanced Options to specify batch options described in Performing a Cluster-Level Rolling Restart. Services that do not support rolling restart will undergo a normal restart, and will not be available during the restart process. If you uncheck the Rolling Upgrade checkbox, Cloudera Manager performs a non-rolling upgrade process. Click Continue.
    10. The Command Progress screen displays the result of the commands run by the wizard as it shuts down all services, activates the new parcel, upgrades services as necessary, deploys client configuration files, and restarts services. Click Continue.
    11. The Host Inspector runs and displays the CDH version running on the hosts. Click Continue.
    12. The wizard reports the result of the upgrade. Click Finish to return to the Home page.
  • Minor upgrade
    1. Log into the Cloudera Manager Admin console.
    2. From the Home tab Status page, click next to the cluster name and select Upgrade Cluster. The Upgrade Wizard starts.
    3. If the option to pick between packages and parcels displays, click the Use Parcels radio button.
    4. In the Choose CDH Version (Parcels) field, select the CDH version. If there are no qualifying parcels, click the click here link to go to the Parcel Configuration Settings page where you can add the locations of parcel repositories. Click Continue.
    5. Read the notices for steps you must complete before upgrading, click the Yes, I ... checkboxes after completing the steps, and click Continue.
    6. Cloudera Manager checks that hosts have the correct software installed. Click Continue.
    7. The selected parcels are downloaded and distributed. Click Continue.
    8. The wizard displays the Perform rolling upgrade to the cluster screen.
    9. Leaving the Rolling Upgrade checkbox checked indicates that Cloudera Manager should upgrade services and perform a rolling restart. Click Advanced Options to specify batch options described in Performing a Cluster-Level Rolling Restart. Services that do not support rolling restart will undergo a normal restart, and will not be available during the restart process. If you uncheck the Rolling Upgrade checkbox, Cloudera Manager performs a non-rolling upgrade process. Click Continue.
    10. The Command Progress screen displays the result of the commands run by the wizard as it shuts down all services, activates the new parcel, upgrades services as necessary, deploys client configuration files, and restarts services. Click Continue.
    11. The Host Inspector runs and displays the CDH version running on the hosts. Click Continue.
    12. The wizard reports the result of the upgrade. Click Finish to return to the Home page.

Remove CDH 5 Packages

If your previous installation of CDH was done using packages, remove those packages on all hosts on which you installed the parcels and refresh the symlinks so that clients will run the new software versions. Skip this step if your previous installation was using parcels.

  1. Uninstall the CDH packages on each host:
    • Not including Impala and Search
      Operating System Command
      RHEL
      $ sudo yum remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
      SLES
      $ sudo zypper remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
      Ubuntu or Debian
      $ sudo apt-get purge bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
    • Including Impala and Search
      Operating System Command
      RHEL
      $ sudo yum remove 'bigtop-*' hue-common impala-shell solr-server sqoop2-client
      SLES
      $ sudo zypper remove 'bigtop-*' hue-common impala-shell solr-server sqoop2-client
      Ubuntu or Debian
      $ sudo apt-get purge 'bigtop-*' hue-common impala-shell solr-server sqoop2-client
  2. Restart all the Cloudera Manager Agents to force an update of the symlinks to point to the newly installed components on each host:
    $ sudo service cloudera-scm-agent restart

Finalize HDFS Rolling Upgrade (CDH 5.2 and higher)

If upgraded from 5.0 or 5.1 to 5.2 or higher, finalize the rolling upgrade in HDFS:
  1. Go to the HDFS service.
  2. Select Actions > Finalize Rolling Upgrade and click Finalize Rolling Upgrade to confirm.