This is the documentation for Cloudera Manager 5.0.0.
Documentation for other versions is available at Cloudera Documentation.

Installation Path A - Automated Installation by Cloudera Manager

If your cluster meets the requirements for Installation Path A, follow the instructions in this section for automated installation by Cloudera Manager. The requirements for Path A are:
  • Uniform SSH access to cluster hosts on the same port from Cloudera Manager Server host.
  • You must log in to the Cloudera Manager Server host using a root account or an account that has password-less sudo permission. See Cloudera Manager Requirements for more information.
  • All hosts must have access to standard package repositories.
  • All hosts must have access to the either archive.cloudera.com on the internet or to a local repository with the necessary installation files.

The general steps in the procedure for Installation Path A follow.

  1. Before You Begin
  2. Download and Run the Cloudera Manager Installer
  3. Configure Databases
  4. Start the Cloudera Manager Admin Console
  5. Use the Cloudera Manager Wizard for Software Installation and Configuration
  6. Change the Default Administrator Password
  7. Test the Installation

Before You Begin

In certain circumstances you may need to perform optional installation and configuration steps.

Install and Configure External Databases

The Cloudera Manager configuration, as well as the other monitoring and management information is stored in databases. If you intend to use an external database for monitoring and management information, install and configure them following the instructions in MySQL Database, Oracle Database, and External PostgreSQL Database.

(CDH 5 only) Install Python 2.6 or 2.7

Python 2.6 or 2.7 is required to run Hue. RHEL 5 and CentOS 5, in particular, require the EPEL repository package.

In order to install packages from the EPEL repository, first download the appropriate repository rpm packages to your machine and then install Python using yum. For example, use the following commands for RHEL 5 or CentOS 5:
$ su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm'
...
$ yum install python26

Configure an HTTP Proxy

The Cloudera Manager installer accesses archive.cloudera.com by using yum on RHEL systems, zypper on SLES systems, or apt-get on Debian/Ubuntu systems. If your hosts access the Internet through an HTTP proxy, you can configure yum, zypper, or apt-get, system-wide, to access archive.cloudera.com through a proxy. To do so, modify the system configuration on the Cloudera Manager Server host and on every cluster host where you want to install CDH as follows:

On Red Hat systems, add the following property to /etc/yum.conf:

proxy=http://server:port/

On SUSE systems, add the following property to /root/.curlrc:

--proxy=http://server:port/

On Debian/Ubuntu systems, add the following property to /etc/apt/apt.conf:

Acquire::http::Proxy "http://server:port";

Download and Run the Cloudera Manager Installer

  1. Download cloudera-manager-installer.bin from Cloudera Downloads to the host where you want to install the Cloudera Manager Server. The host must be on your cluster or accessible to your cluster over your network. Install Cloudera Manager on a single host.
  2. Change cloudera-manager-installer.bin to have executable permission.
    $ chmod u+x cloudera-manager-installer.bin
  3. Run cloudera-manager-installer.bin.
      Note: The installer's default behavior is to install the Cloudera Manager packages from the Internet. If you have created a local repository and configured your host to recognize that repository, you can instruct the installer to use local repositories by running the cloudera-manager-installer.bin with the --skip_repo_package=1 option.
    $ sudo ./cloudera-manager-installer.bin
  4. Read the Cloudera Manager Readme and then press Enter to choose Next.
  5. Read the Cloudera Manager License and then press Enter to choose Next. Use the arrow keys and press Enter to choose Yes to confirm you accept the license.
  6. Read the Oracle Binary Code License Agreement and then press Enter to choose Next. Use the arrow keys and press Enter to choose Yes to confirm you accept the Oracle Binary Code License Agreement. The Cloudera Manager installer begins installing the Oracle JDK and the Cloudera Manager repo files and then installs the packages. The installer also installs the Cloudera Manager Server.
      Note: If an error message "Failed to start server" appears while running cloudera-manager-installer.bin, exit the installation program. If the Cloudera Manager Server log file /var/log/cloudera-scm-server/cloudera-scm-server.log contains the following message, then it's likely you have SELinux enabled:
    Caused by: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
            at java.net.URLClassLoader$1.run(Unknown Source)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(Unknown Source)
            at java.lang.ClassLoader.loadClass(Unknown Source)
            ...
    You can disable SELinux by running the following command on the Cloudera Manager Server host:
    $ sudo setenforce 0
    To disable it permanently, edit /etc/selinux/config.
  7. Note the complete URL provided for the Cloudera Manager Admin Console, including the port number, which is 7180 by default. Press Enter to choose OK to continue.
  8. Press Enter to choose OK to exit the installer.
  Note:

Configure Databases

If you are using an external database, prepare the database as described in Cloudera Manager Server External Database.

Start the Cloudera Manager Admin Console

The Cloudera Manager Server URL takes the following form http://Server host:port, where Server host is the fully-qualified domain name or IP address of the host where the Cloudera Manager Server is installed and port is the port configured for the Cloudera Manager Server. The default port is 7180.
  1. In a web browser, enter the URL, including the port, for the Cloudera Manager Server. The login screen for Cloudera Manager Admin Console displays.
  2. Log into Cloudera Manager Admin Console. The default credentials are: Username: admin Password: admin. Cloudera Manager does not support changing the admin username for the installed account. You can change the password using Cloudera Manager after you run the installation wizard. While you cannot change the admin username, you can add a new user, assign administrative privileges to the new user, and then delete the default admin account.

Use the Cloudera Manager Wizard for Software Installation and Configuration

The following instructions describe how to use the Cloudera Manager installation wizard to do an initial installation and configuration. The wizard lets you:

  • Select the version of Cloudera Manager you want to install
  • Find the cluster hosts you specify via hostname and IP address ranges
  • Connect to each host with SSH to install the Cloudera Manager Agent and other components
  • Install the Oracle JDK on the cluster hosts (if not already installed)
  • Install CDH and managed service packages or parcels
  • Configure CDH and managed services automatically and start the services
  Important: All hosts in the cluster must have some way to access installation files via one of the following methods:
  • Internet access to allow the wizard to install software packages or parcels from archive.cloudera.com.
  • A custom internal repository that the host(s) can access. For example, for a Red Hat host, you could set up a Yum repository. See Creating and Using a Local Package Repository for more information about this option.

Choose Cloudera Manager Edition and Hosts

  1. Choose which edition to install:
    • Cloudera Express, which does not require a license, but provides a somewhat limited set of features
    • Cloudera Enterprise Data Hub Edition Trial, which does not require a license, but expires after 60 days and cannot be renewed
    • Cloudera Enterprise with one of the following license types:
      • Basic Edition
      • Flex Edition
      • Data Hub Edition
    If you choose Cloudera Express or Cloudera Enterprise Data Hub Edition Trial, you can elect to upgrade the license at a later time. See Managing Licenses.
  2. If you have elected Cloudera Enterprise, install a license:
    1. Click Upload License.
    2. Click the document icon to the left of the Select a License File text field.
    3. Navigate to the location of your license file, click the file, and click Open.
    4. Click Upload.
  3. Information is displayed indicating what the CDH installation includes. At this point, you can access online Help or the Support Portal if you wish. Click Continue to proceed with the installation.
  4. To enable Cloudera Manager to automatically discover your cluster hosts where you want to install CDH and managed services, enter the cluster hostnames or IP addresses. You can also specify hostname and IP address ranges. For example:
    Range Definition Matching Hosts
    10.1.1.[1-4] 10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
    host[1-3].company.com host1.company.com, host2.company.com, host3.company.com
    host[07-10].company.com host07.company.com, host08.company.com, host09.company.com, host10.company.com

    You can specify multiple addresses and address ranges by separating them by commas, semicolons, tabs, or blank spaces, or by placing them on separate lines. Use this technique to make more specific searches instead of searching overly wide ranges. The scan results will include all addresses scanned, but only scans that reach hosts running SSH will be selected for inclusion in your cluster by default. If you don't know the IP addresses of all of the hosts, you can enter an address range that spans over unused addresses and then deselect the hosts that do not exist (and are not discovered) later in this procedure. However, keep in mind that wider ranges will require more time to scan.

  5. Click Search. Cloudera Manager identifies the hosts on your cluster to allow you to configure them for services. If there are a large number of hosts on your cluster, wait a few moments to allow them to be discovered and shown in the wizard. If the search is taking too long, you can stop the scan by clicking Abort Scan. To find additional hosts, click New Search, add the host names or IP addresses and click Search again. Cloudera Manager scans hosts by checking for network connectivity. If there are some hosts where you want to install services that are not shown in the list, make sure you have network connectivity between the Cloudera Manager Server host and those hosts. Common causes of loss of connectivity are firewalls and interference from SELinux.
  6. Verify that the number of hosts shown matches the number of hosts where you want to install services. Deselect host entries that do not exist and deselect the hosts where you do not want to install services. Click Continue. The Select Repository page displays.

Choose Software Installation Method

  1. Select the repository type you want to use for the installation: parcels or packages.
    • Use Parcels:
      1. Choose the parcels you want to install. The choices you see depend on the repositories you have chosen – a repository may contain multiple parcels. Only the parcels for the latest supported service versions are configured by default.
        You can add additional parcels for previous versions by specifying custom repositories. For example, you can find the locations of the previous CDH 4 parcels at http://archive.cloudera.com/cdh4/parcels/. Or, if you are installing CDH 4.3 and want to use Sentry for Hive Authorization, you can add the Sentry parcel using this mechanism. To add a custom parcel repository:
        1. Enter the URL of the repository you want into More Options field, and click the + Add button. The URL you specify here will also be added to the list of repositories listed in the Configuring Server Parcel Settings page and a parcel will be added to the list of parcels on the Select Repository page. If you have multiple repositories configured, you will see all the unique parcels contained in all your repositories.
    • Use Packages:
      1. Select the major release of CDH to install.
      2. Select the specific release of CDH to install.
      3. Select the specific releases of Impala and Solr to install, assuming you have selected an appropriate CDH version. You can choose either the latest version or use a custom repository. Choose None if you do not want to install that service.
  2. Select the specific release of Cloudera Manager Agent to install on your hosts. You may choose either the version that matches with the Cloudera Manager Server you are currently using or you can specify an installation at a custom repository.
  3. If you opted to use custom repositories for installation files, you can provide a GPG key URL that will apply for all repositories.
  4. Click Continue.

Provide Credentials for Authenticating with Hosts

  1. Select root or enter the user name for an account that has password-less sudo permissions.
  2. Select an authentication method:
    • If you choose to use password authentication, enter and confirm the password.
    • If you choose to use public-key authentication provide a passphrase and path to the required key files.
    • You can choose to specify an alternate SSH port. The default value is 22.
    • You can specify the maximum number of host installations to run at once. The default value is 10.
  3. Click Continue.

Installation of JDK, Cloudera Manager Agents, CDH, and Managed Services on Hosts

  1. Cloudera Manager performs the following:
    • Parcels - installs the Oracle JDK and the Cloudera Manager Agent packages and starts the Agent. During the parcel installation, progress is indicated for the two phases of the parcel installation process (Download and Distribution) in a separate progress bars. If you are installing multiple parcels you will see progress bars for each parcel. When the Continue button appears at the bottom of the screen, the installation process is completed.
    • Packages - configures package repositories, installs the Oracle JDK, CDH and managed service and the Cloudera Manager Agent packages, and starts the Agent. When the Continue button appears at the bottom of the screen, the installation process is completed. If the installation has completed successfully on some hosts but failed on others, you can click Continue if you want to skip installation on the failed hosts and continue to the next screen to start configuring services on the successful hosts.
    While software is being installed, the status of installation on each host is displayed. You can click the Details link for individual hosts to view detailed information about the installation and error messages if installation fails on any hosts. If you click the Abort Installation button while installation is in progress, it will halt any pending or in-progress installations and roll back any in-progress installations to a clean state. The Abort Installation button does not affect host installations that have already completed successfully or already failed.
  2. The Host Inspector runs to validate the installation, and provides a summary of what it finds, including all the versions of the installed components. If the validation is successful, click Continue.

Add Services

  1. In the first page of the Add Services wizard you choose the combination of services to install and whether to install Cloudera Navigator:
    • Click the radio button next to the combination of services to install:
      CDH 4 CDH 5
      • Core Hadoop - HDFS, MapReduce, ZooKeeper, Oozie, Hive, and Hue
      • Core with HBase
      • Core with Impala
      • All Services - HDFS, MapReduce, ZooKeeper, HBase, Impala, Oozie, Hive, Hue, and Sqoop
      • Custom Services - Any combination of services.
      • Core Hadoop - HDFS, YARN (includes MapReduce 2), ZooKeeper, Oozie, Hive, Hue, and Sqoop
      • Core with HBase
      • Core with Impala
      • Core with Search
      • Core with Spark
      • All Services - HDFS, YARN (includes MapReduce 2), ZooKeeper, Oozie, Hive, Hue, Sqoop, HBase, Impala, Solr, Spark, and Keystore Indexer
      • Custom Services - Any combination of services.
      As you select the services, keep the following in mind:
      • Some services depend on other services; for example, HBase requires HDFS and ZooKeeper. Cloudera Manager tracks dependencies and installs the correct combination of services.
      • In a CDH 4 cluster, the MapReduce service is the default MapReduce computation framework. Choose Custom Services to install YARN or use the Add Service functionality to add YARN after installation completes.
          Important: You can create a YARN service in a CDH 4 cluster, but it is not considered production ready.
      • In a CDH 5 cluster, the YARN service is the default MapReduce computation framework. Choose Custom Services to install MapReduce or use the Add Service functionality to add MapReduce after installation completes.
          Important: In CDH 5 the MapReduce service has been deprecated. However, the MapReduce service is fully supported for backward compatibility through the CDH 5 life cycle.
      • The Flume service can be added only after your cluster has been set up.
    • If you have chosen Data Hub Edition Trial or Cloudera Enterprise, optionally check the Include Cloudera Navigator checkbox to enable Cloudera Navigator. See the Cloudera Navigator Documentation.
    Click Continue. The Customize Role Assignments page displays.
  2. Customize the assignment of role instances to hosts. The wizard evaluates the hardware configurations of the hosts to determine the best hosts for each role. The wizard assigns all worker roles to the same set of hosts to which the HDFS DataNode role is assigned. These assignments are typically acceptable, but you can reassign services to hosts of your choosing, if desired.

    Click a field below a role to display a dialog containing a pageable list of hosts. If you click a field containing multiple hosts, you can also select All Hosts to assign the role to all hosts or Custom to display the pageable hosts dialog.

    The following shortcuts for specifying host names are supported:
    • Range of hostnames (without the domain portion)
      Range Definition Matching Hosts
      10.1.1.[1-4] 10.1.1.1, 10.1.1.2, 10.1.1.3, 10.1.1.4
      host[1-3].company.com host1.company.com, host2.company.com, host3.company.com
      host[07-10].company.com host07.company.com, host08.company.com, host09.company.com, host10.company.com
    • IP addresses
    • Rack name

    Click the View By Host button for an overview of the role assignment by host ranges.

  3. When you are satisfied with the assignments, click Continue. The Database Setup page displays.
  4. Configure settings for required databases:
    1. Choose the database type:
      • Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure all required databases. Make a note of the auto-generated passwords.
      • Select Use Custom Databases to specify external databases. Enter the database host, database type, database name, username, and password for the databases that you created when you set up databases for Cloudera Manager.
    2. Click Test Connection to confirm that Cloudera Manager can communicate with the databases using the information you have supplied. If the test succeeds in all cases, click Continue; otherwise check and correct the information you have provided for the databases and then try the test again. (For Hive, if you are using the embedded database, you will see a message saying the database will be created at a later point in the installation process.) The Review Changes page displays.
  5. Review the configuration changes to be applied. Confirm the settings entered for file system paths. The file paths required vary based on the services to be installed. For example, you might confirm the NameNode Data Directory and the DataNode Data Directory for HDFS. Click Continue. The wizard starts the services.
  6. When all of the services are started, click Continue. You will see a success message indicating that your cluster has been successfully started.
  7. Click Continue to proceed to the Home Page.

Change the Default Administrator Password

As soon as possible after running the wizard and beginning to use Cloudera Manager, you should change the default administrator password:
  1. Right-click the logged-in username at the far right of the top navigation bar and select Change Password.
  2. Enter the current password, and a new password twice and then click Submit.

Test the Installation

You can test the installation following the instructions in Testing the Installation.