Upgrading Cloudera Manager 5 Using Packages

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

This topic describes how to upgrade Cloudera Manager 5.x using packages. These steps apply to both minor and maintenance upgrades. When you upgrade Cloudera Manager using packages, you run operating system package commands from the command line, and then complete the upgrade using Cloudera Manager.

You can also manually upgrade Cloudera Manager using tarballs.

In most cases it is possible to upgrade Cloudera Manager without shutting down most CDH services, although you might need to stop some dependent services. CDH daemons can continue running, unaffected, while Cloudera Manager is upgraded. The upgrade process does not affect your CDH installation. After upgrading Cloudera Manager, you might also want to upgrade CDH 4 clusters to CDH 5, or upgrade to a more recent minor version of CDH.

The Cloudera Manager upgrade process does the following:
  • Upgrades the database schema to reflect the current version.
  • Upgrades the Cloudera Manager Server and all supporting services.
  • Upgrades the Cloudera Manager agent.
  • Redeploys client configurations to ensure that client services have the most current configuration.
Steps to upgrade Cloudera Manager 5 using packages:
Step Description Link
1 Collect the information you will need to upgrade Cloudera Manager. This includes user accounts, passwords, database URLs and other items. You must gather this information before beginning the upgrade because some of the information is only available from the Cloudera Manager Admin Console, which will not be available during the upgrade process. Step 1: Collect Upgrade Information
2 Complete the pre-upgrade steps and review some special warnings about upgrades. Step 2: Complete Pre-Upgrade Steps
3 Back up the Cloudera Manager databases. Step 3: Back Up Cloudera Manager Databases
4 If your Cloudera Manager hosts are using an unsupported version of the JDK, you must upgrade to a supported version of the JDK before upgrading Cloudera Manager. If you also plan to upgrade CDH, you must also upgrade the JDK on all cluster hosts. Step 4:

Upgrading to Oracle JDK 1.7

Upgrading to Oracle JDK 1.8

5

If the Cloudera Manager host does not have access to the internet, or you wish to install a version other than the latest version of Cloudera Manager, you configure access to the Cloudera Manager software, either from the Cloudera public repository or a local package repository that you create.

Step 5: Creating and Using a Package Repository for Cloudera Manager
6

If you are upgrading from Cloudera Navigator 2.6 or lower, you must upgrade the Cloudera Navigator data management component. Cloudera Navigator 2.6 is supported on Cloudera Manager 5.7.x and higher.

Step 6: Upgrading the Cloudera Navigator Data Management Component
7 Upgrade the Cloudera Manager server and agent software. Step 7: Upgrade the Cloudera Manager Server
8 Verify and test your upgrade. Step 8: Verify and Test the Upgrade
9 Upgrade any required Cloudera Navigator components:
  • Cloudera Manager Key Trustee Server
  • Cloudera Navigator Key HSM
  • Cloudera Navigator Key Trustee KMS
  • Cloudera Navigator Encrypt.

The Cloudera Navigator Data Management Component is upgraded automatically when you upgrade Cloudera Manager.

Step 9: Upgrading Cloudera Navigator Components
10 (Optional) Upgrade CDH. You are not required to upgrade CDH after upgrading Cloudera Manager. You can upgrade CDH at a later time. Step 10: (Optional) Upgrade CDH

Step 1: Collect Upgrade Information

Before starting an upgrade of Cloudera Manager, collect the following information:
  • Host credentials. You must have SSH access and be able to log in using a root account or an account that has password-less sudo permission.
  • The version of Cloudera Manager used in your cluster. Go to Support > About.
  • The version of CDH. The CDH version number displays next to the cluster name on the Home page.
  • The services enabled in your cluster. Go to Clusters > Cluster name.
  • Whether the cluster was installed using parcels or packages. This information displays next to the CDH version on the Home page of Cloudera Manager.
  • The version of the JDK deployed in the cluster. Go to Support > About.
  • Whether or not you have enabled Cloudera Navigator auditing.
  • For all databases used in your cluster:
    • Type of database (PostgreSQL, Embedded PostgreSQL, MySQL, MariaDB, or Oracle)
    • Hostnames of the databases
    • Credentials for the databases
    To locate database information:
    • Sqoop, Oozie, and Hue – Go to Cluster Name > Configuration > Database Settings.

    • Hive Metastore – Go to the Hive service, select Configuration, and select the Hive Metastore Database category.
    • Sentry – Go to the Sentry service, select Configuration, and select the Sentry Server Database category.
  • Operating system version.
  • Whether the cluster uses AES-256 encryption.

Step 2: Complete Pre-Upgrade Steps

Before beginning a Cloudera Manager upgrade, do the following:

  1. Review the system requirements for the new versions you are upgrading to.

    See CDH 5 and Cloudera Manager 5 Requirements and Supported Versions.

  2. If you are upgrading from Cloudera Manager 5.4.x to Cloudera Manager 5.5 or higher, perform the following steps required for the associated Cloudera Navigator upgrade:
    1. Stop the Navigator Metadata Server role.
    2. Back up the Navigator Metadata Server storage directory.
    3. Make sure that the Navigator Metadata Server has sufficient memory to complete the upgrade.
    4. If you are using an Oracle database, in SQL*Plus, ensure that the following additional privileges are set:
      GRANT EXECUTE ON sys.dbms_crypto TO nav;
      GRANT CREATE VIEW TO nav;
      where nav is the user of the Navigator Audit Server database.
    For further information, see Upgrading the Cloudera Navigator Data Management Component.
  3. Note the following:
    • Cloudera Management Service TLS/SSL configuration

      If you have enabled TLS security for the Cloudera Manager Admin Console, as of Cloudera Manager 5.1, Cloudera Management Service roles try to communicate with Cloudera Manager using TLS, and fail to start until TLS/SSL properties have been configured.

    • Navigator

      If you have enabled auditing with Cloudera Navigator, during the upgrade to Cloudera Manager 5, auditing is suspended and is only restarted when you restart the roles of audited services. You will be instructed to stop some services in a later step.

  4. If you have previously installed Kafka 1.2, and are upgrading from Cloudera Manager 5.4 or lower, remove the Kafka CSD:
    1. Determine the location of the CSD directory:
      1. Select Administration > Settings.
      2. Click the Custom Service Descriptors category.
      3. Retrieve the directory from the Local Descriptor Repository Path property.
    2. Delete the Kafka CSD from the directory.
  5. Review package (RPM) dependencies. A Cloudera Manager upgrade may introduce new package dependencies. If your organization has restrictions or requires prior approval for installation of packages, see the list of Package Dependencies before upgrading Cloudera Manager.

Step 3: Back Up Cloudera Manager Databases

Back up the following Cloudera Manager databases:
  • Cloudera Navigator Audit Server
  • Cloudera Navigator Metadata Server
  • Activity Monitor
  • Reports Manager
You can look up the hostnames, credentials, and other information about these databases by going to Clusters > Cloudera Management Service > Configuration and selecting the Database category.
  1. Stop the Cloudera Management Service:
    1. Select Clusters > Cloudera Management Service.
    2. Select Actions > Stop.
  2. Back up databases. See Backing Up Databases for detailed instructions for each supported type of database.
  3. Start the Cloudera Management Service:
    1. Select Clusters > Cloudera Management Service.
    2. Select Actions > Start.

Step 4: Upgrade the JDK

If your Cloudera Manager hosts are using an unsupported version of the JDK, you must upgrade to a supported version of the JDK before upgrading Cloudera Manager. If you also plan to upgrade CDH, you must also upgrade the JDK on all cluster hosts.

If you have enabled TLS/SSL, you must reinstall CA certificates to your truststores after upgrading the JDK. See Creating Truststores.

Step 5: Establish Access to the Software

If the Cloudera Manager host does not have access to the internet, or you wish to install a version other than the latest version of Cloudera Manager, you configure access to the Cloudera Manager software, either from the Cloudera public repository or a local package repository that you create.

See Creating and Using a Package Repository for Cloudera Manager.

Step 6: Upgrade the Cloudera Navigator Data Management Component

If you are upgrading from Cloudera Navigator 2.6 or lower, you must upgrade the Cloudera Navigator data management component. Cloudera Navigator 2.6 is supported on Cloudera Manager 5.7.x and higher.

See Upgrading the Cloudera Navigator Data Management Component.

Step 7: Upgrade the Cloudera Manager Server

  1. If your cluster is running the embedded PostgreSQL database, stop all services that are using the embedded database. These may include:
    • Hive service and all services such as Impala and Hue that use the Hive metastore
    • Oozie
    • Sentry
  2. If your cluster is running the Cloudera Navigator data management component and the following services are enabled for auditing, stop the indicated roles:
    • HDFS - NameNode:
    • HBase - Master and RegionServers
    • Hive - HiveServer2
    • Hue - Beeswax Server
    Stopping these roles renders any service depending on these roles unavailable. For the HDFS - NameNode case this implies most of the services in the cluster will be unavailable until the upgrade is finished.
  3. Stop the Cloudera Management Service:
    1. Select Clusters > Cloudera Management Service.
    2. Select Actions > Stop.
  4. Stop Cloudera Manager Server, Database, and Agent:
    1. Use the Cloudera Manager Admin Console to stop any running commands. These include commands a user runs and commands Cloudera Manager automatically triggers in response to a state change or a schedule. You can either wait for commands to complete or abort any running commands. For more information on viewing and aborting running commands, see Viewing Running and Recent Commands.
    2. On the host running the Cloudera Manager Server, stop the Cloudera Manager Server:
      sudo service cloudera-scm-server stop
    3. On the host where the database runs, (usually the Cloudera Manager Server host) stop the database:
      sudo service cloudera-scm-server-db stop
    4. If the Cloudera Manager host is also running the Cloudera Manager Agent, stop the Cloudera Manager Agent:
      sudo service cloudera-scm-agent stop
  5. Back up the following directories on the Cloudera Manager server host:
    • /etc/cloudera-scm-server
    • /etc/cloudera-scm-agent
  6. If you are using JDK 1.6, upgrade the JDK on Cloudera Manager Server JDK 1.7 or 1.8. After you install JDK 7, Cloudera Manager can install JDK 7 on all of the cluster hosts. To upgrade to JDK 1.8, you must manually install the JDK on all cluster hosts. Follow the steps in Java Development Kit Installation, and then return here to continue the upgrade.
  7. Establish access to the Cloudera Manager Server packages. You can either upgrade from the Cloudera repository at https://archive.cloudera.com/cm5/, or you can create your own package repository, as described in Creating and Using a Package Repository for Cloudera Manager. You must create your own repository if Cloudera Manager does not have Internet access or you want to upgrade to a version of Cloudera Manager other than the latest version.

    To upgrade using the Cloudera repository:

    1. Download the Cloudera .repo file for your distribution by starting at https://archive.cloudera.com/cm5/ and navigating to the directory that matches your operating system.

      For example, for Red Hat or CentOS 6, you would go to https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/. In that directory, find the repo file that contains information including the repository base URL and GPG key. The contents of the cloudera-manager.repo are similar to the following:

      [cloudera-manager]
      # Packages for Cloudera Manager, Version 5, on RHEL or CentOS 6 x86_64
      name=Cloudera Manager
      baseurl=https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5/
      gpgkey = https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/RPM-GPG-KEY-cloudera
      gpgcheck = 1
      For Ubuntu or Debian systems, go to the appropriate release directory, for example, https://archive.cloudera.com/cm4/debian/wheezy/amd64/cm. The repo file, in this case, cloudera.list, is similar to the following:
      # Packages for Cloudera Manager, Version 5, on Debian 7.0 x86_64
      deb https://archive.cloudera.com/cm5/debian/wheezy/amd64/cm wheezy-cm5 contrib
      deb-src https://archive.cloudera.com/cm5/debian/wheezy/amd64/cm wheezy-cm5 contrib
      For example:
      $ wget https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/cloudera-manager.repo
    2. Do one of the following on the Cloudera Manager Server host:
      • Upgrade to the most recent version of Cloudera Manager:
        copy the cloudera-manager.repo file to the configuration location for the package management software for your system:
        RHEL
        Copy cloudera-manager.repo to /etc/yum.repos.d/
        SLES
        Copy cloudera-manager.repo to /etc/zypp/repos.d/
        Ubuntu or Debian
        Copy cloudera.list to /etc/apt/sources.list.d/
      • Upgrade to an earlier version of Cloudera Manager:
        RHEL-compatible or SLES
        1. Edit the cloudera-manager.repo file to change the baseurl to point to the version of Cloudera Manager you want to download. For example, to install Cloudera Manager version 5.0.1, change:

          baseurl=https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5/

          to:

          baseurl=https://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.0.1/.

        2. Save the edited file:
          • For RHEL or CentOS, save it in /etc/yum.repos.d/.
          • For SLES, save it in /etc/zypp/repos.d.
        Ubuntu or Debian
        1. Download the Cloudera Manager list file (cloudera.list) using the links provided at Cloudera Manager Version and Download Information. For example, for Ubuntu 10.04 (lucid), this file is located at

          https://archive.cloudera.com/cm5/ubuntu/lucid/amd64/cm/cloudera.list.

        2. Edit the cloudera.list file to change the second-to-last element to specify the version of Cloudera Manager you want to install. For example, with Ubuntu lucid, if you want to install Cloudera Manager version 5.0.1, change:

          deb https://archive.cloudera.com/cm5/ubuntu/lucid/amd64/cm lucid-cm5 contrib

          to:

          deb https://archive.cloudera.com/cm5/ubuntu/lucid/amd64/cm lucid-cm5.0.1 contrib.

        3. Save the edited file in the directory /etc/apt/sources.list.d/.
    3. Run the following commands:
      RHEL
      sudo yum clean all
      sudo yum upgrade cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent
      SLES
      sudo zypper clean --all
      sudo zypper up -r https://archive.cloudera.com/cm5/sles/11/x86_64/cm/5/
      To download from your own repository:
      sudo zypper clean --all
      sudo zypper rr cm
      sudo zypper ar -t rpm-md http://myhost.example.com/path_to_cm_repo/cm
      sudo zypper up -r http://myhost.example.com/path_to_cm_repo
      Ubuntu or Debian
      The following commands clean cached repository information and update Cloudera Manager components:
      sudo apt-get clean
      sudo apt-get update
      sudo apt-get dist-upgrade
      sudo apt-get install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent

      If the Cloudera Manager instance you are upgrading uses the embedded PostgreSQL database, add cloudera-manager-server-db-2 to the list of packages in the apt-get install command. The embedded PostgreSQL database should not be used in production environments.

      During this process, you might be prompted about your configuration file version:
      Configuration file `/etc/cloudera-scm-agent/config.ini'
      ==> Modified (by you or by a script) since installation.
      ==> Package distributor has shipped an updated version.
      What would you like to do about it ? Your options are:
      Y or I : install the package maintainer's version
      N or O : keep your currently-installed version
      D : show the differences between the versions
      Z : start a shell to examine the situation
      The default action is to keep your current version.

      You will receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.

  8. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is moved to a file with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.
  9. On the Cloudera Manager Server host, verify that you now have the following packages, corresponding to the version of Cloudera Manager you installed, by running the following command:
    RPM-based distributions
    $ rpm -qa 'cloudera-manager-*'
    cloudera-manager-repository-5.8-1.noarch
    cloudera-manager-server-5.9.0-0.cm590.p0.252.el6.x86_64
    cloudera-manager-agent-5.9.0-0.cm590.p0.252.el6.x86_64
    cloudera-manager-daemons-5.9.0-0.cm590.p0.252.el6.x86_64
    Ubuntu or Debian
    ~# dpkg-query -l 'cloudera-manager-*'
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name                   Version                Description
    +++-======================-======================-============================================================
    ii  cloudera-manager-agent 5.9.0-0.cm590.p0.252~sq The Cloudera Manager Agent
    ii  cloudera-manager-daemo 5.9.0-0.cm590.p0.252~sq Provides daemons for monitoring Hadoop and related tools.
    ii  cloudera-manager-serve 5.9.0-0.cm590.p0.252~sq The Cloudera Manager Server
    You might also see an entry for the cloudera-manager-server-db-2 if you are using the embedded PostgreSQL database, and additional packages for plug-ins, depending on what was previously installed on the server host. If the cloudera-manager-server-db-2 package is installed, and you do not plan to use the embedded database, you can remove this package.
  10. Start Cloudera Manager Server. On the Cloudera Manager Server host (the host on which you installed the cloudera-manager-server package) do the following:
    1. If you are using the embedded PostgreSQL database for Cloudera Manager, start the database:
      sudo service cloudera-scm-server-db start
    2. Start the Cloudera Manager Server:
      sudo service cloudera-scm-server start
      You should see the following:
      Starting cloudera-scm-server:                              [  OK  ]
  11. Log in to the Cloudera Manager Admin Console.

    The Upgrade Wizard displays.

  12. Upgrade the Agent software on cluster hosts manually or by using Cloudera Manager:
    • Cloudera Manager installs Agent software
      1. Select Yes, I would like to upgrade the Cloudera Manager Agent packages now and click Continue.
      2. Select the release of the Cloudera Manager Agent to install. Normally, this is the Matched Release for this Cloudera Manager Server. However, if you used a custom repository (instead of archive.cloudera.com) for the Cloudera Manager server, select Custom Repository and provide the required information. The custom repository allows you to use an alternative location, but that location must contain the matched Agent version. See Creating and Using a Package Repository for Cloudera Manager.
      3. Click Continue. The JDK Installation Options page displays.
        • If you want Cloudera Manager to install JDK 1.7 on all cluster hosts, select Install Oracle Java SE Development Kit (JDK) .
        • If local laws permit you to deploy unlimited strength encryption, and you are running a secure cluster, select the Install Java Unlimited Strength Encryption Policy Files checkbox.
      4. Click Continue.
      5. Specify credentials and initiate Agent installation:
        • Select root or enter the username for an account that has password-less sudo permission.
        • Select an authentication method:
          • If you choose password authentication, enter and confirm the password.
          • If you choose public-key authentication, provide a passphrase and path to the required key files.
        • You can specify an alternate SSH port. The default value is 22.
        • You can specify the maximum number of host installations to run at once. The default value is 10.
      6. Click Continue.

        The Cloudera Manager Agent packages and, if selected, the JDK are installed.

      7. Click Continue.

        The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If there are problems, you can make changes and then rerun the inspector.

        When you are satisfied with the inspection results, click Continue.

    • Manually install Agent software
      1. On all cluster hosts except the Cloudera Manager Server host, stop the Agent:
        sudo service cloudera-scm-agent stop
      2. In the Cloudera Admin Console, select No, I would like to skip the agent upgrade now and click Continue.
      3. Copy the appropriate repo file as described in step 7.
      4. Run the following commands on all hosts except the Cloudera Manager Server host:
        RHEL
        sudo yum clean all
        sudo yum upgrade cloudera-manager-daemons cloudera-manager-agent
        SLES
        sudo zypper clean --all
        sudo zypper up -r https://archive.cloudera.com/cm5/sles/11/x86_64/cm/5/
        To download from your own repository:
        sudo zypper clean --all
        sudo zypper rr cm
        sudo zypper ar -t rpm-md http://myhost.example.com/path_to_cm_repo/cm
        sudo zypper up -r http://myhost.example.com/path_to_cm_repo
        Ubuntu or Debian
        Use the following commands to clean cached repository information and update Cloudera Manager components:
        sudo apt-get clean
        sudo apt-get update
        sudo apt-get dist-upgrade
        sudo apt-get install cloudera-manager-agent cloudera-manager-daemons

        If the Cloudera Manager instance you are upgrading uses the embedded PostgreSQL database, add cloudera-manager-server-db-2 to the list of packages in the apt-get install command. The embedded PostgreSQL database should not be used in production environments.

        During this process, you might be prompted about your configuration file version:
        Configuration file '/etc/cloudera-scm-agent/config.ini'
        ==> Modified (by you or by a script) since installation.
        ==> Package distributor has shipped an updated version.
        What would you like to do about it ? Your options are:
        Y or I : install the package maintainer's version
        N or O : keep your currently-installed version
        D : show the differences between the versions
        Z : start a shell to examine the situation
        The default action is to keep your current version.

        You will receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.

      5. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is moved to a file with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.
      6. On all cluster hosts, start the Agent:
        sudo service cloudera-scm-agent start
      7. Click Continue. The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If there are problems, you can make changes and then rerun the inspector. When you are satisfied with the inspection results, click Continue.
  13. Click Finish.
  14. If you are upgrading from Cloudera Manager 5.0 and are using an external database for Cloudera Navigator, the Database Setup page displays. Configure these database settings:
    1. Enter the database host, database type, database name, username, and password for the database that you created when you set up the database.
    2. Click Test Connection to confirm that Cloudera Manager can communicate with the database using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the database and then try the test again. (For some servers, if you are using the embedded database, you will see a message saying the database will be created at a later step in the installation process.)
  15. The Review Changes page displays. Review the configuration changes to be applied and click Continue. The Upgrade wizard displays a dialog box allowing you to choose whether to restart the Cloudera Management Service.
  16. Click Continue.

    If you keep the default selection, the Upgrade wizard restarts the Cloudera Management Service.

  17. Click Finish.

    The Home page displays.

    All services (except for any services you stopped) should now be running.

  18. If, as part of this upgrade, you stopped some selected services and roles, restart the following roles:
    • HDFS - NameNode
    • HBase - Master and RegionServers
    • Hive - HiveServer2
    • Hue - Beeswax Server
  19. If you upgraded the JDK do the following:
    1. If the Cloudera Manager Server host is also running a Cloudera Manager Agent, restart the Cloudera Manager Server:
      sudo service cloudera-scm-server restart

      If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems.

    2. Restart all services:
      1. From the Home > Status tab click next to the cluster name and select Restart.
      2. In the confirmation dialog box that displays, click Restart.
  20. If Cloudera Manager reports stale configurations after the upgrade, restart the cluster services and redeploy the client configurations (If you are planning to also upgrade CDH, this step is not required):
    1. On the Home > Status tab, click next to the cluster name and select Restart.
    2. In the confirmation dialog box, click Restart.
    3. On the Home > Status tab, click next to the cluster name and select Deploy Client Configuration.
    4. In the confirmation dialog box, click Deploy Client Configuration.

Step 8: Verify and Test the Upgrade

If the commands to update and start the Cloudera Manager Server complete without errors, you can assume the upgrade has completed successfully. To verify, you can check that the server versions have been updated.
  1. In the Cloudera Manager Admin Console, click the Hosts tab.
  2. Click Host Inspector. On large clusters, the host inspector may take some time to finish running. You must wait for the process to complete before proceeding to the next step.
  3. Click Show Inspector Results. All results from the host inspector process are displayed, including the currently installed versions. If this includes listings of current component versions, the installation completed as expected.
  4. Verify that the monitoring features are working as expected; follow the instructions in Testing the Installation.

Step 9: Upgrade any Required Navigator Components

Step 10: (Optional) Upgrade CDH

Cloudera Manager 5 can manage both CDH 4 and CDH 5, so upgrading existing CDH 4 and 5 installations is not required, but you may want to upgrade to the latest version. For more information on upgrading CDH, see Upgrading CDH and Managed Services Using Cloudera Manager.