Cloudera Upgrade

This topic provides an overview of upgrade procedures for Cloudera Manager and CDH. The procedures described here use Cloudera Manager to perform some or all of the upgrade steps. You can also upgrade unmanaged CDH clusters (clusters that are not managed by Cloudera Manager). See Upgrading Unmanaged CDH Using the Command Line.

There are two major parts of an upgrade: upgrading Cloudera Manager and upgrading CDH. When upgrading Cloudera Manager, you can select either an upgrade that uses tarballs or uses operating system packages. When you upgrade CDH, you can select an upgrade that uses packages or parcels. As part of an upgrade, you might also need to install a new version of the JDK. Cloudera Navigator is also upgraded as part of the Cloudera Manager upgrade.

It is not required that you upgrade both Cloudera Manager and CDH, but you must upgrade to versions of Cloudera Manager and CDH that are compatible. Cloudera Manager can manage clusters for the current and previous major versions of CDH and any equal or lower minor version of CDH. For example, Cloudera Manager 5.7.1 can manage clusters with CDH 5.7.2, CDH 5.6.1, and CDH 4.8.6 but cannot manage a cluster with CDH 5.8.1. Cloudera Manager 5.7.1 cannot manage clusters using CDH 3.x.

There are several options you can choose as you proceed through the upgrade process:

Assessing the Impact of an Upgrade

You should plan for a sufficient maintenance window to perform an upgrade. Depending on which components you are upgrading, the number of hosts in your cluster, and the type of hardware, you might need up to a full day to perform all the steps to upgrade your cluster. There are also some information-gathering steps you should perform before beginning the upgrade procedures. These steps are detailed in the Cloudera Manager and CDH upgrade procedures.

Before upgrading, consult the release notes for the relevant versions of Cloudera Manager and CDH that you intend to upgrade to learn about API changes, deprecated features, new features, and incompatible changes. Also check the CDH 5 and Cloudera Manager 5 Requirements and Supported Versions page to make sure that you are using a supported operating system, JDK, database, and other components.

There are three types of upgrades: major, minor, and maintenance:
Major Upgrades
Major upgrades bring large changes to functionality and update Hadoop to more recent versions. There might be incompatible changes in data formats and significant changes and additions to the user interface in Cloudera Manager. You can only upgrade from one major version to the next major version, for example from version 4.8.1 to version 5.8.0. To be ready for the next major version upgrade, Cloudera recommends that you upgrade to a 5.x version.

There might be database schema changes for Cloudera Manager that are automatically handled by the upgrade process. Client configurations are redeployed.

Significant down time may be required for major upgrades.

Minor Upgrades
Minor upgrades upgrade your software to a higher minor version of a major release, for example from version 5.4.x to version 5.8.x. These upgrades can introduce new functionality and bug fixes. Incompatible changes or changes to data formats are generally not introduced in minor upgrades. There might be database schema changes for Cloudera Manager that are automatically handled by the upgrade process. Client configurations are redeployed.
Maintenance Upgrades
Maintenance upgrades are used only to fix critical bugs or address security issues. No new functionality or incompatible changes are introduced.

Upgrading Cloudera Manager

Cloudera Manager 5 Upgrade

To upgrade Cloudera Manager, you perform the following tasks:
  1. Upgrade the Cloudera Manager server software on the Cloudera Manager host using operating system package commands from the command line (for example, yum on RHEL systems). You can also manually upgrade Cloudera Manager using tarballs. Tarball upgrades are more suitable for environments where there are existing tools to manage software distributed over many hosts. Cloudera Manager automates much of this process and is the recommend means for upgrading and managing your CDH clusters.
  2. Upgrade the Cloudera Manager agent software on all cluster hosts. The Cloudera Manager upgrade wizard can upgrade the agent software (and, optionally, the JDK), or you can install the agent and JDK software manually from tarballs. The CDH software is not upgraded during this process.

To upgrade Cloudera Manager, see Upgrading Cloudera Manager.

Upgrading CDH

CDH Parcel Upgrades

CDH Package Upgrades

CDH upgrades contain updated versions of the Hadoop software and other components. You can use Cloudera Manager to upgrade CDH using either parcels or packages. Parcels are the preferred method because they are managed by Cloudera Manager, which automatically downloads, distributes, and activates the correct versions of the software. If you use parcels and have enabled high availability for HDFS, you can perform a rolling upgrade to upgrade CDH without cluster down time. For an easier upgrade experience, consider switching from packages to parcels so that Cloudera Manager can automate more of the process. You can also switch from packages to parcels when upgrading CDH 5.

Cloudera Manager 5.3 introduced an enhanced CDH upgrade wizard that supports major (CDH 4 to CDH 5), minor (CDH 5.x to 5.y), and maintenance upgrades (CDH a.b.x to CDH a.b.y). Both parcels and package installations are supported, but packages must be manually installed, whereas parcels are installed by Cloudera Manager.

See Upgrading CDH and Managed Services Using Cloudera Manager

Upgrading Cloudera Navigator Components

Cloudera Navigator Metadata and Audit servers are automatically upgraded when you upgrade Cloudera Manager. You can also optionally upgrade other Cloudera Navigator components such as Cloudera Navigator Key Trustee Server, Cloudera Navigator Key HSM, and Cloudera Navigator Encrypt. You do not have to upgrade these components along with Cloudera Manager or CDH upgrades.

See Upgrading Cloudera Navigator Components.

Upgrading the JDK

Before upgrading Cloudera Manager or CDH, ensure that all cluster hosts are using a supported version of the Oracle Java Development Kit (JDK). All cluster hosts must use the same version of the JDK. See:

Upgrading with the EMC DSSD D5

To upgrade a cluster that uses the DSSD D5 storage appliance, you first upgrade Cloudera Manager, if needed, and then you use Cloudera Manager to upgrade CDH. See Upgrading with the EMC DSSD D5.