Upgrading the Cloudera Manager Agents

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

Loading Filters ... 6.0.0 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0 6.0.0 5.15 5.14 5.13 5.12 5.11 5.10 5.9 5.8 5.7 5.6 5.5 5.4 5.3 5.2 5.1 5.0

Upgrade the Cloudera Manager Agents

You can upgrade the agents using one of the two options below.

Option 1. Upgrade the Agents using Cloudera Manager (Recommended)

After the Cloudera Manager server starts and you log in to the Cloudera Manager Admin Console, the Upgrade Cloudera Manager page displays. (It may take several minutes for the server to start.)

If the Upgrade Cloudera Manager page does not display after you upgraded the packages on the Cloudera Manager server host, open the following URL in your web browser:

https://my_cloudera_manager_server_host:port/cmf/upgrade
https://my_cloudera_manager_server_host:port/cmf/upgrade-wizard/welcome

The status of Agent upgrades is displayed in one or more groups. Because you might have only upgraded the Cloudera Manager agent on the Cloudera Manager server host, the first group shows that host as having an upgraded agent. If the hosts managed by Cloudera Manager have different operating systems, a group for each operating system displays the Agent upgrade status for those hosts.

Follow the instructions on the upgrade page to upgrade all the agents.

  1. Click Yes, I would like to upgrade Cloudera Manager Agent packages now.
  2. Click Continue.

    The Upgrade Cloudera Manager Agent Packages page displays.

  3. If there are more than one group of hosts that require agent upgrades, select the group from the drop-down list labeled Upgrade Cloudera Manager Agent Packages running on:. If there is only one group that requires upgrades, this drop-down list does not appear.

    In Cloudera Manager 5.15 or higher, Cloudera Manager can upgrade agents even when they are running on different operating systems or versions (One OS group at a time).

  4. Click Upgrade Cloudera Manager Agent packages.

    The Upgrade Cloudera Manager Agent Packages page displays.

  5. If you are using a local package repository instead of the public repository at https://archive.cloudera.com, select the Custom Repository option when installing the Cloudera Manager Agent packages and enter the Custom Repository URL.
    RHEL / CentOS

    Use the baseurl value in the cloudera-manager.repo file as the Custom Repository. Use the gpgkey value as the Custom GPG Key URL.

    SLES

    Use the baseurl value in the cloudera-manager.repo file as the Custom Repository. Use the gpgkey value as the Custom GPG Key URL.

    Debian / Ubuntu

    Use the entire deb url contrib line from the cloudera_Manager.list file as the Custom Repository. Use the url/archive.key as the Custom GPG Key URL.

    Use the Package Repository URL/cm6/6.0.x as the Custom Repository.

  6. Click Continue.

    The Accept JDK License page displays.

  7. If you want to install JDK 78 on all hosts, select Install Oracle Java SE Development Kit.
  8. Click Continue.

    The Enter Login Credentials page displays.

  9. Specify the credentials and initiate Agent installation:
    1. Select root for the root account, or select Another user and enter the username for an account that has password-less sudo permission.
    2. Select an authentication method:
      • If you choose the All hosts accept same password option, enter and confirm the password.
      • If you choose the All hosts accept same private key option, provide a passphrase and path to the required key files.
    3. Modify the default SSH port if necessary.
    4. Specify the maximum Number of Simultaneous Installations to run at once. The default and recommended value is 10. Adjust this parameter based on your network capacity.
  10. Click Continue.

    The Cloudera Manager Agent packages and, if selected, the JDK are installed.

  11. When the installations complete, click Finish.

    The Upgrade Cloudera Manager page displays the status of the upgrade.

    If there are additional groups of hosts that require Agent upgrades, select the next group from the Upgrade Cloudera Manager Agent Packages running on: drop-down list, and repeat the agent installation steps.

  12. Click Run Host Inspector to run the host inspector. Inspect the output and correct any warnings. If problems occur, you can make changes and then rerun the inspector.
  13. When you are satisfied with the inspection results, start the Cloudera Management Service.
  14. Click the link at the bottom of the page to go back to the Home Page.
  15. Click Continue.

    The Cloudera Manager Agent packages and, if selected, the JDK are installed.

    The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then click Run Again to rerun the inspector.

  16. Click Continue.

    The Review Changes page displays and suggests configuration changes you may need to make.

  17. Make any necessary changes and click Continue.

    The Upgrade Wizard restarts the Cloudera Manager Management Service.

  18. Click Finish.
  19. The Cloudera Manager Home page opens and displays the status of the cluster. It can take several minutes for all of the services to display their current status. You may need to restart some services or redeploy stale client configurations.
Option 2. Upgrade the Agents using the Command Line
  • If the Cloudera Manager hosts have internet access, you can use the publicly available repositories from https://archive.cloudera.com. Fill in the form at the top of this page to generate the contents of the repository file for your system. The package manager uses this file to download and install a new version of the Cloudera Manager software. You will copy the contents of this file to your clipboard and then create the file on the hosts in a later step.
  • If the Cloudera Manager hosts do not have internet access, configure a local package repository hosted on your network, replace the default repository URL below, and click Apply to update the contents of the repository file in the text box below. You will copy the contents of this file to your clipboard and then create the file on the hosts in a later step. For example: http://MyWebServer:1234/cloudera-repos

Package Repository URL:

Perform the following commands on all hosts managed by Cloudera Manager:

(You can also multiplex the same set of commands to all hosts by using utilities such as csshX, pdsh, or pssh.)
  1. Log in to each host using ssh. For example:
    ssh host1.example.com
  2. Remove any older files in the existing repository directory:
    RHEL / CentOS
    sudo rm /etc/yum.repos.d/cloudera*manager.repo*
    SLES
    sudo rm /etc/zypp/repos.d/cloudera*manager.repo*
    Debian / Ubuntu
    sudo rm /etc/apt/sources.list.d/cloudera*.list*
  3. RHEL / CentOS

    Create a file named /etc/yum.repos.d/cloudera-manager.repo with the following content:

    [cloudera-manager]
    # Packages for Cloudera Manager
    name=Cloudera Manager
    baseurl=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.15
    gpgkey=https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera
    gpgcheck=1
    SLES

    Create a file named /etc/zypp/repos.d/cloudera-manager.repo with the following content:

    [cloudera-manager]
    # Packages for Cloudera Manager
    name=Cloudera Manager
    baseurl=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/5.15
    gpgkey=https://archive.cloudera.com/cm5/sles/12/x86_64/cm/RPM-GPG-KEY-cloudera
    gpgcheck=1
    Debian / Ubuntu
    Create a file named /etc/apt/sources.list.d/cloudera_manager.list with the following content:
    # Packages for Cloudera Manager
    deb https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
    deb-src https://archive.cloudera.com/cm5/debian/jessie/amd64/cm/ jessie-cm5.15 contrib
    sudo apt-get update
  4. Stop the Cloudera Manager Agent.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl stop cloudera-scm-agent
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-agent stop
  5. Install JDK 1.8.
    RHEL / CentOS
    sudo yum install oracle-j2sdk1.8.x86_64
    SLES
    sudo zypper install oracle-j2sdk1.8.x86_64
    Debian / Ubuntu
    sudo apt-get install oracle-j2sdk1.8
  6. Upgrade the agent packages.

    Note: Only add cloudera-manager-server-db-2 if you are using the embedded PostgreSQL database.
    RHEL / CentOS
    sudo yum clean all
    sudo yum repolist
    sudo yum upgrade cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    SLES
    sudo zypper clean --all
    sudo zypper up cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    Debian / Ubuntu
    sudo apt-get clean
    sudo apt-get update
    sudo apt-get dist-upgrade
    sudo apt-get install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server-db-2
    You might be prompted about your configuration file version:
    Configuration file '/etc/cloudera-scm-agent/config.ini'
    ==> Modified (by you or by a script) since installation.
    ==> Package distributor has shipped an updated version.
    What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
    D : show the differences between the versions
    Z : start a shell to examine the situation
    The default action is to keep your current version.
    You may receive a similar prompt for /etc/cloudera-scm-server/db.properties. Answer N to both prompts.
  7. If you customized the /etc/cloudera-scm-agent/config.ini file, your customized file is renamed with the extension .rpmsave or .dpkg-old. Merge any customizations into the /etc/cloudera-scm-agent/config.ini file that is installed by the package manager.
  8. Verify that you have the correct packages installed.

    Debian / Ubuntu
    dpkg-query -l 'cloudera-manager-*'
    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name                   Version                Description
    +++-======================-======================-============================================================
    ii  cloudera-manager-agent 5.15.0-0.cm...~sq The Cloudera Manager Agent
    ii  cloudera-manager-daemo 5.15.0-0.cm...~sq Provides daemons for monitoring Hadoop and related tools.
    RHEL / CentOS / SLES
    rpm -qa 'cloudera-manager-*'
    cloudera-manager-agent-5.15.0-..cm...
    cloudera-manager-daemons-5.15.0-..cm...
    cloudera-manager-server-db-2-5.15.0-..cm...
  9. If you are using the embedded PostgreSQL database, start the database:
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-server-db
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-server-db start
  10. Start the Cloudera Manager Agent.
    RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
    sudo systemctl start cloudera-scm-agent
    If the agent starts without errors, no response displays.
    RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
    sudo service cloudera-scm-agent start
    You should see the following:
    Starting cloudera-scm-agent: [ OK ]
  11. In Cloudera Manager 5.15 or higher, you can monitor the progress at https://my_cloudera_manager_server_host:port/cmf/upgrade.
  12. When all the agents are upgraded, run the Host Inspector.
  13. In Cloudera Manager 5.14 or lower, Run the Run Host Inspector: Select Hosts > All Hosts and click Inspect All Hosts.
  14. The Host Inspector runs to inspect your managed hosts for correct versions and configurations. If problems occur, you can make changes and then rerun the inspector.
  15. The Cloudera Manager Home page opens and displays the status of the cluster. It can take several minutes for all of the services to display their current status. You may need to restart some services or redeploy stale client configurations.
If you are upgrading from Cloudera Manager 5.5.0 or lower to Cloudera Manager 5.5.0 or higher, hard restart the agent on all hosts to update and restart the supervisord process.
RHEL 7, SLES 12, Debian 8, Ubuntu 16.04
sudo systemctl stop supervisord
sudo systemctl start cloudera-scm-agent
RHEL 5 or 6, SLES 11, Debian 6 or 7, Ubuntu 12.04, 14.04
sudo service cloudera-scm-agent hard_restart