This is the documentation for Cloudera Manager 5.1.x.
Documentation for other versions is available at Cloudera Documentation.

Performing a Rolling Upgrade on a Cluster

Required Role:

  Important: This feature is available only with a Cloudera Enterprise license.
For other licenses, the following applies:
  • Cloudera Express - the feature is not available.
  • Cloudera Enterprise Data Hub Edition Trial - the feature will not be available after you end the trial or the trial license expires.
To obtain a license for Cloudera Enterprise, please fill in this form or call 866-843-7207. After you install a Cloudera Enterprise license, the feature will be available.

Cloudera Manager's rolling upgrade feature takes advantage of parcels and the HDFS High Availability configuration to enable you to upgrade your cluster software and restart the upgraded services, without taking the entire cluster down. You must have HDFS High Availability enabled to perform a rolling upgrade.

You can perform a rolling upgrade between minor versions of CDH 4, or between minor versions of CDH 5, except Beta versions, as they become available. It is not possible to perform a rolling upgrade from CDH 4 to CDH 5 because of incompatibilities between the two major versions. Instead, follow the instructions for a full upgrade at Upgrading from CDH 4 to CDH 5 Parcels.

A rolling upgrade involves two steps:
  1. Download, distribute, and activate the parcel for the new software you want to install.
  2. Perform a rolling restart to restart the services in your cluster. You can do a rolling restart of individual services, or if you have High Availability enabled, you can perform a restart of the entire cluster. Cloudera Manager will manually fail over your NameNode at the appropriate point in the process so that your cluster will not be without a functional NameNode.
To avoid lots of alerts during the upgrade process, you can enable maintenance mode on your cluster before you start the upgrade. This will stop email alerts and SNMP traps from being sent, but will not stop checks and configuration validations from being made. Be sure to exit maintenance mode when you have finished the upgrade in order to re-enable Cloudera Manager alerts.

The steps to perform a rolling upgrade of a cluster are as follows.

Ensure High Availability is Enabled

To enable High Availability see HDFS High Availability for instructions. You do not need to enable Automatic Failover for rolling restart to work, though you can enable it if you wish. Automatic Failover does not affect the rolling restart operation. If you have JobTracker High Availability configured, Cloudera Manager will fail over the JobTracker during the rolling restart, but this is not a requirement for performing a rolling upgrade.

Download, Distribute, and Activate Parcels

  1. In the Cloudera Manager Admin Console, click the Parcels indicator in the top navigation bar ( or ) to go to the Parcels page.
  2. In the parcels page, click Download for the version(s) you want to download. If the parcel you want is not shown here — for example, you want to upgrade to version of CDH that is not the most current version — you can make additional parcel repos available through the parcel settings page.
    • CDH 4 - If you want to run both CDH, Cloudera Impala, and Cloudera Search you should download the CDH, Impala, and Solr parcels. You can find the locations of the previous CDH 4 parcels at http://archive.cloudera.com/cdh4/parcels/. If you are upgrading to CDH 4.3 and want to use Sentry, you can find the separate Sentry parcel at http://archive.cloudera.com/sentry/parcels/latest/). Sentry is included with CDH 4.4 or later parcels.
    • CDH 5 - Download the CDH parcel.
    If your Cloudera Manager server does not have Internet access, you can obtain the required parcel file(s) and put them into the local repository. See Creating and Using a Parcel Repository for more details.
  3. When the download has completed, click Distribute for the version you downloaded.
  4. When the parcel has been distributed and unpacked, the button will change to say Activate. If you are doing a major version upgrade (that is, from CDH 4 to CDH 5 using parcels), after the distribution phase the button will be labeled Upgrade rather than Activate. In this case, follow the instructions at Upgrading from CDH 4 to CDH 5 Parcels.
  5. Click Activate. You are asked if you want to restart the cluster. Do not restart the cluster at this time.
  6. Click Close.

Upgrade the Hive Metastore Database

Required if you are upgrading from an earlier version of CDH 4 to CDH 4.2 or later.

  1. Make a backup copy of your Hive metastore database.
  2. Go to the Hive service.
  3. Select Actions > Stop and click Stop to confirm.
  4. Select Actions > Upgrade Hive Metastore Database Schema and click Upgrade Hive Metastore Database Schema to confirm.
  5. If you have multiple instances of Hive, perform the upgrade on each metastore database.

Upgrade the Oozie ShareLib

  1. Go to the Oozie service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Select Actions > Install Oozie ShareLib and click Install Oozie ShareLib to confirm.
  4. When the command completes, click Close.

Upgrade Sqoop

  1. Go to the Sqoop service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Select Actions > Upgrade Sqoop and click Upgrade Sqoop to confirm.
  4. When the command completes, click Close.

Restart the Services

  1. On the Home page, click to the right of the cluster name and select Restart.
  2. Click the Restart button in the confirmation pop-up that appears. The Command Details window shows the progress of starting services.

Remove CDH Packages

If your previous installation of CDH was done using packages, you must remove those packages on all hosts on which you installed the parcels and refresh the symlinks so that clients will run the new software versions. Skip this step if you installed the previous version of CDH 5 using parcels.

  • CDH 5 -
    1. Uninstall the CDH packages on each host:
      • Not including Impala and Search
        Operating System Command
        RHEL
        $ sudo yum remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
        SLES
        $ sudo zypper remove bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
        Ubuntu or Debian
        $ sudo apt-get purge bigtop-utils bigtop-jsvc bigtop-tomcat hue-common sqoop2-client
      • Including Impala and Search
        Operating System Command
        RHEL
        $ sudo yum remove 'bigtop-*' hue-common impala-shell solr-server sqoop2-client
        SLES
        $ sudo zypper remove 'bigtop-*' hue-common impala-shell solr-server sqoop2-client
        Ubuntu or Debian
        $ sudo apt-get purge 'bigtop-*' hue-common impala-shell solr-server sqoop2-client
  • CDH 4 -
    1. If Hue is configured to use SQLite as its database:
      1. Stop the Hue service.
      2. Back up the desktop.db to a temporary location before deleting the old Hue Common package. The location of the database can be found in the Hue service Configuration tab under Service > Database > Hue's Database Directory.
        Important: Removing the Hue Common package will remove your Hue database; if you do not back it up you may lose all your Hue user account information.
    2. Uninstall the CDH packages on each host:
      • Not including Impala and Search
        Operating System Command
        RHEL
        $ sudo yum remove hadoop hue-common bigtop-jsvc bigtop-tomcat
        SLES
        $ sudo zypper remove hadoop hue-common bigtop-jsvc bigtop-tomcat
        Ubuntu or Debian
        $ sudo apt-get purge hadoop hue-common bigtop-jsvc bigtop-tomcat
      • Including Impala and Search
        Operating System Command
        RHEL
        $ sudo yum remove hadoop hue-common impala-shell solr-server 'bigtop-*'
        SLES
        $ sudo zypper remove hadoop hue-common impala-shell solr-server 'bigtop-*'
        Ubuntu or Debian
        $ sudo apt-get purge hadoop hue-common impala-shell solr-server 'bigtop-*'

Update Symlinks for the Newly Installed Components

Restart all the Cloudera Manager Agents to force an update of the symlinks to point to the newly installed components on each host:
$ sudo service cloudera-scm-agent restart

Restore Hue Database

If you removed CDH 4 packages, restore the Hue database from the backup.
  1. In the Cloudera Manager Admin Console, select the Hue service.
  2. Select Actions > Stop and click Stop to confirm.
  3. Copy the backup to the newly created Hue database directory: /opt/cloudera/parcels/CDH-4.x.0-x.cdh4.x.0.p0.xx/share/hue/desktop.
  4. Restart the Hue service.