Upgrading the Cloudera Manager Server

This topic provides procedures for backing up the Cloudera Manager Server.

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

Loading Filters ... 6.0.1 6.0.0 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.0.1 6.0.0 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0

Establish Access to the Software

Cloudera Manager needs access to a package repository that contains the updated software packages.

  • If the Cloudera Manager hosts have internet access, you can use the publicly available repositories from https://archive.cloudera.com. Fill in the form at the top of this page to generate the contents of the repository file for your system. The package manager uses this file to download and install a new version of the Cloudera Manager software. You will copy the contents of this file to your clipboard and then create the file on the hosts in a later step.
  • If the Cloudera Manager hosts do not have internet access, configure a local package repository hosted on your network, replace the default repository URL below, and click Apply to update the contents of the repository file in the text box below. You will copy the contents of this file to your clipboard and then create the file on the hosts in a later step. For example: http://MyWebServer:1234/cloudera-repos

Package Repository URL:

  1. Log in to the Cloudera Manager Server host.
    ssh my_cloudera_manager_server_host
  2. Remove any older files in the existing repository directory:
    RHEL / CentOS
    sudo rm /etc/yum.repos.d/cloudera*manager.repo*
    SLES
    sudo rm /etc/zypp/repos.d/cloudera*manager.repo*
    Debian / Ubuntu
    sudo rm /etc/apt/sources.list.d/cloudera*.list*
  3. Fill in the form at the top of this page.
  4. Create a repository file so that the package manager can locate and download the binaries:
    RHEL / CentOS

    Create a file named /etc/yum.repos.d/cloudera-manager.repo with the following content:

    [cloudera-manager]
    # Packages for Cloudera Manager
    name=Cloudera Manager
    baseurl=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15
    gpgkey=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera
    gpgcheck=1
    SLES

    Create a file named /etc/zypp/repos.d/cloudera-manager.repo with the following content:

    [cloudera-manager]
    # Packages for Cloudera Manager
    name=Cloudera Manager
    baseurl=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/5.15
    gpgkey=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/RPM-GPG-KEY-cloudera
    gpgcheck=1
    Debian / Ubuntu
    Create a file named /etc/apt/sources.list.d/cloudera_manager.list with the following content:
    # Packages for Cloudera Manager
    deb https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
    deb-src https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
    sudo apt-get update

    The repository file, as created, refers to the most recent maintenance release of the specified minor release. If you would like to use a specific maintenance version, for example 5.15.1, replace 5.15 with 5.15.1 in the generated repository file shown above.

  5. A Cloudera Manager upgrade can introduce new package dependencies. Your organization may have restrictions or require prior approval for installation of new packages. You can determine which packages may be installed or upgraded:

    RHEL / CentOS
    yum deplist cloudera-manager-agent
    SLES
    zypper info --requires cloudera-manager-agent
    Debian / Ubuntu
    apt-cache depend cloudera-manager-agent

Install JDK 8

Oracle JDK 1.8 is required on all cluster hosts managed by Cloudera Manager 6.0.0 or higher. If JDK 1.8 is already installed on your hosts, skip the steps in this section.

If you are upgrading to Cloudera Manager 6.0.0 or higher, you can manually install JDK 8 on the Cloudera Manager server host, and then, as part of the Cloudera Manager upgrade process, you can specify that Cloudera Manager upgrade the JDK on the remaining hosts.

  1. Log in to the Cloudera Manager Server host.
    ssh my_cloudera_manager_server_host
  2. Install JDK 1.8.
    RHEL / CentOS
    sudo yum install oracle-j2sdk1.8.x86_64
    SLES
    sudo zypper install oracle-j2sdk1.8.x86_64
    Debian / Ubuntu
    sudo apt-get install oracle-j2sdk1.8
  3. Open the following file in a text editor:
    /etc/default/cloudera-scm-server
  4. Edit the line that begins with export JAVA_HOME (if this line does not exist, add it) and change the path to the path of the new JDK (you can find the path under /usr/java).
    For example: (RHEL and SLES)
    export JAVA_HOME="/usr/java/jdk1.8.0_141-cloudera"
    For example: (Ubuntu)
    export JAVA_HOME="/usr/lib/jvm/java-8-oracle-cloudera"
  5. Save the file.

Upgrade the Cloudera Manager Server

  1. Log in to the Cloudera Manager Server host.
    ssh my_cloudera_manager_server_host
  2. If your cluster is running the embedded PostgreSQL database, stop all services that are using the embedded database. These can include:
    • Hive service and all services such as Impala and Hue that use the Hive metastore
    • Oozie
    • Sentry
    • Sqoop
  3. Stop the Cloudera Management Service.
    1. Log in to the Cloudera Manager Admin Console.
    2. Select Clusters > Cloudera Management Service.
    3. Select Actions > Stop.
  4. Ensure that you have disabled any scheduled replication or snapshot jobs and wait for any running commands from the Cloudera Manager Admin Console to complete before proceeding with the upgrade.
  5. Stop the Cloudera Manager Server.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-server
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server stop
  6. If you are using the embedded PostgreSQL database, stop the Cloudera Manager Embedded PostgreSQL database:
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-server-db
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server-db stop

    If you are not using the embedded PostgreSQL database and you attempt to stop it, you might see a message indicating that the service cannot be found. If you see a message that the shutdown failed, then the embedded database is still running, probably because services are connected to the Hive metastore. If the database shutdown fails due to connected services, issue the following command:

    RHEL-compatible 7 and higher, Ubuntu 16.04
    sudo service cloudera-scm-server-db next_stop_fast
    sudo service cloudera-scm-server-db stop
    All other Linux distributions
    sudo service cloudera-scm-server-db fast_stop
  7. Stop the Cloudera Manager Agent.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-agent
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-agent stop
  8. Upgrade the packages. Include the cloudera-manager-server-db-2 package in the command only if you are using the embedded PostgreSQL database.
    RHEL / CentOS
    sudo yum clean all
    sudo yum upgrade cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    SLES
    sudo zypper clean --all
    sudo zypper up cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    Debian / Ubuntu
    sudo apt-get clean
    sudo apt-get update
    sudo apt-get dist-upgrade
    sudo apt-get install cloudera-manager-server cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2

    You might be prompted about your configuration file version:

    Configuration file '/etc/cloudera-scm-agent/config.ini'
    ==> Modified (by you or by a script) since installation.
    ==> Package distributor has shipped an updated version.
    What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
    D : show the differences between the versions
    Z : start a shell to examine the situation
    The default action is to keep your current version.

    You may receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.

    You may be prompted to accept the GPG key. Answer y.

    Retrieving key from https://archive.cloudera.com/.../cm/RPM-GPG-KEY-cloudera
    Importing GPG key ...
     Userid     : "Yum Maintainer <webmaster@cloudera.com>"
     Fingerprint: ...
     From       : https://archive.cloudera.com/.../RPM-GPG-KEY-cloudera
  9. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.
  10. Verify that you have the correct packages installed.

    Debian / Ubuntu
    dpkg-query -l 'cloudera-manager-*'
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name                   Version                Description
    +++-======================-======================-============================================================
    ii  cloudera-manager-agent 5.15.0-0.cm...~sq The Cloudera Manager Agent
    ii  cloudera-manager-daemo 5.15.0-0.cm...~sq Provides daemons for monitoring Hadoop and related tools.
    ii  cloudera-manager-serve 5.15.0-0.cm...~sq The Cloudera Manager Server
    
    RHEL / CentOS / SLES
    rpm -qa 'cloudera-manager-*'
    cloudera-manager-server-5.15.0-..cm...
    cloudera-manager-agent-5.15.0-..cm...
    cloudera-manager-daemons-5.15.0-..cm...
    cloudera-manager-server-db-2-5.15.0-..cm...
  11. If you are using the embedded PostgreSQL database, start the database:
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-server-db
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server-db start
  12. Start the Cloudera Manager Agent.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-agent
    If the agent starts without errors, no response displays.
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-agent start
    You should see the following:
    Starting cloudera-scm-agent: [ OK ]
  13. Start the Cloudera Manager Server.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-server
    If the Cloudera Manager Server starts without errors, no response displays.
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server start
    You should see the following:
    Starting cloudera-scm-server: [ OK ]
  14. Use a Web browser to open the Cloudera Manager Admin Console using the following URL:
    http://cloudera_Manager_server_hostname:7180/cmf/upgrade
    http://cloudera_Manager_server_hostname:7180/cmf/upgrade-wizard/welcome
    It can take several minutes for the Cloudera Manager Server to start, and the Cloudera Manager Admin Console is unavailable until the server startup is complete and the Upgrade Cloudera Manager page displays. Continue with the steps on the next page to upgrade the Cloudera Manager Agents.

To complete the Cloudera Manager upgrade, continue with Upgrading the Cloudera Manager Agents.