Cloudera Manager 5.1.1

A Single Interface to Manage Your CDH Cluster

Cloudera Manager is a unified management interface that makes it easy to install, configure, and manage a CDH cluster. It automatically ships with Cloudera Enterprise or Cloudera Express to help you get up and running with Hadoop faster.

Cloudera Manager 5.1.1 works with both CDH 4 and CDH 5 and is available with:
Cloudera Express - Easily deploy, manage, monitor and perform diagnostics on your Hadoop cluster
Cloudera Enterprise – Includes all the above capabilities, plus advanced management features and support including zero downtime upgrades and backup and disaster recovery.

Instructions for Installing Pervious Cloudera Manager Versions

To install any previous version of Cloudera Manager 5, you must create a custom repository for the RPM or DEB Files.

See the instructions in Installing Older Versions of Cloudera Manager 5.

The download locations of the 5.0.x RPM or DEB Files are listed in Cloudera Manager Version and Download Information.

Known Issues and Workarounds in Cloudera Manager 5.1.1

The following sections describe the current known issues in Cloudera Manager 5.1.1.

Could not find a healthy host with CDH5 on it to create HiveServer2 error during upgrade

When upgrading from CDH 4 to CDH 5, if no parcel is active then the error message "Could not find a healthy host with CDH5 on it to create HiveServer2" displays. This can happen when transitioning from packages to parcels, or if you explicitly deactivate the CDH 4 parcel (which is not necessary) before upgrade.

Workaround: Wait 30 seconds and retry the upgrade.

AWS Installation wizard requires Java 7u45 to be installed on Cloudera Manager Server host

Cloudera Manager 5.1 installs Java 7u55 by default. However, the AWS installation wizard does not work with Java 7u55 due to a bug in the jClouds version packaged with Cloudera Manager.

Workaround:
  1. Stop the Cloudera Manager Server.
    $ sudo service cloudera-scm-server stop 
  2. Uninstall Java 7u55 from the Cloudera Manager Server host.
  3. Install Java 7u45 (which you can download from http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html#jdk-7u45-oth-JPR) on the Cloudera Manager Server host.
  4. Start the Cloudera Manager Server.
    $ sudo service cloudera-scm-server start
  5. Run the AWS installation wizard.
  Note: Due to a bug in Java 7u45 (http://bugs.java.com/bugdatabase/view_bug.do?bug_id=8014618), SSL connections between the Cloudera Manager Server and Cloudera Manager Agents and between the Cloudera Management Service and CDH processes break intermittently. If you do not have SSL enabled on your cluster, there is no impact.

The Spark History Server does not start when Kerberos authentication is enabled.

The Spark History Server does not start when managed by a Cloudera Manager 5.1 instance when Kerberos authentication is enabled.

Workaround:
  1. Go to the Spark service.
  2. Expand the Service-Wide > Advanced category.
  3. Add the following configuration to the History Server Environment Advanced Configuration Snippet property:
    SPARK_HISTORY_OPTS=-Dspark.history.kerberos.enabled=true \
    -Dspark.history.kerberos.principal=principal \
    -Dspark.history.kerberos.keytab=keytab
where principal is the name of the Kerberos principal to use for the History Server, and keytab is the path to the principal's keytab file on the local filesystem of the host running the History Server.

Spurious warning on Accumulo 1.6 gateway hosts

When using the Accumulo shell on a host with only an Accumulo 1.6 Service gateway role, users will receive a warning about failing to create the directory /var/log/accumulo. The shell works normally otherwise.

Workaround: The warning is safe to ignore.

Accumulo 1.6 service log aggregation and search does not work

Cloudera Manager log aggregation and search features are incompatible with the log formatting needed by the Accumulo Monitor. Attempting to use either the "Log Search" diagnostics feature or the log file link off of an individual service role's summary page will result in empty search results.

Severity: High

Workaround: Operators can use the Accumulo Monitor to see recent severe log messages. They can see recent log messages below the WARNING level via a given role's process page and can inspect full logs on individual hosts by looking in /var/log/accumulo.

Cloudera Manager incorrectly sizes Accumulo Tablet Server max heap size after 1.4.4-cdh4.5.0 to 1.6.0-cdh4.6.0 upgrade

Because the upgrade path from Accumulo 1.4.4-cdh4.5.0 to 1.6.0-cdh4.6.0 involves having both services installed simultaneously, Cloudera Manager will be under the impression that worker hosts in the cluster are oversubscribed on memory and attempt to downsize the max heap size allowed for 1.6.0-cdh4.6.0 Tablet Servers.

Severity: High

Workaround: Manually verify that the Accumulo 1.6.0-cdh4.6.0 Tablet Server max heap size is large enough for your needs. Cloudera recommends you set this value to the sum of 1.4.4-cdh4.5.0 Tablet Server and Logger heap sizes.

If a NodeManager that is used as ApplicationMaster is decommissioned, YARN jobs will hang

Jobs can hang on NodeManager decommission due to a race condition when continuous scheduling is enabled.

Workaround:
  1. Go to the YARN service.
  2. Expand the ResourceManager Default Group > Resource Management category.
  3. Uncheck the Enable Fair Scheduler Continuous Scheduling checkbox.
  4. Click Save Changes to commit the changes.
  5. Restart the YARN service.

The YARN property ApplicationMaster Max Retries has no effect in CDH 5

The issue arises because yarn.resourcemanager.am.max-retries was replaced with yarn.resourcemanager.am.max-attempts.

Workaround:
  1. Add the following to ResourceManager Advanced Configuration Snippet for yarn-site.xml, replacing MAX_ATTEMPTS with the desired maximum number of attempts:
    <property>
    <name>yarn.resourcemanager.am.max-attempts</name><value>MAX_ATTEMPTS</value>
    </property>
  2. Restart the ResourceManager(s) to pick up the change.

Cluster CDH version not configured correctly for package installs

If you have installed CDH as a package, after the install make sure that the cluster CDH version matches the package CDH version. If the cluster CDH version does not match the package CDH version, Cloudera Manager will incorrectly enable and disable service features based on the cluster's configured CDH version.

Workaround:

Manually configure the cluster CDH version to match the package CDH version. Click ClusterName > Actions > Configure CDH Version. In the dialog, Cloudera Manager displays the installed CDH version, and asks for confirmation to configure itself with the new version.

Accumulo installations using LZO do not indicate dependence on the GPL Extras parcel

Accumulo 1.6 installations that use LZO compression functionality do not indicate that LZO depends on the GPL Extras parcel. When Accumulo is configured to use LZO, Cloudera Manager has no way to track that the Accumulo service now relies on the GPL Extras parcel. This prevents Cloudera Manager from warning administrators before they remove the parcel while Accumulo still requires it for proper operation.

Workaround: Check your Accumulo 1.6 service for the configuration changes mentioned in the Cloudera documentation for using Accumulo with CDH prior to removing the GPL Extras parcel. If the parcel is mistakenly removed, reinstall it and restart the Accumulo 1.6 service.

NodeManager Container Memory not imported to YARN correctly

When MapReduce configurations are imported to YARN, the property yarn.nodemanager.resource.memory-mb may not be set correctly for optimal YARN performance.

Workaround: Configure this property manually after running the import, then restart YARN.

Clients of the JobHistory Server Admin Interface Require Advanced Configuration Snippet

Clients of the JobHistory server administrative interface, such as the mapred hsadmin tool, may fail to connect to the server when run on hosts other than the one where the JobHistory server is running.

Workaround: Add the following to both the MapReduce Client Advanced Configuration Snippet for mapred-site.xml and the Cluster-wide Advanced Configuration Snippet for core-site.xml, replacing JOBHISTORY_SERVER_HOST with the hostname of your JobHistory server:
<property>
<name>mapreduce.history.admin.address</name>
<value>JOBHISTORY_SERVER_HOST:10033</value>
</property>

Created pools are not preserved when Dynamic Resource Pools page is used to configure YARN or Impala

Pools created on demand are not preserved when changes are made using the Dynamic Resource Pools page. If the Dynamic Resource Pools page is used to configure YARN and/or Impala services in a cluster, it is possible to specify pool placement rules that create a pool if one does not already exist. If changes are made to the configuration using this page, pools created as a result of such rules are not preserved across the configuration change.

Workaround: Submit the YARN application or Impala query as before, and the pool will be created on demand once again.

Reset non-default HDFS File Block Storage Location Timeout value after upgrade from CDH 4 to CDH 5

During an upgrade from CDH 4 to CDH 5, if the HDFS File Block Storage Locations Timeout was previously set to a custom value, it will now be set to 10 seconds or the custom value, whichever is higher. This is required for Impala to start in CDH 5, and any value under 10 seconds is now a validation error. This configuration is only emitted for Impala and no services should be adversely impacted.

Workaround: None.

If installing CDH 4 packages, the Impala 1.3.0 option does not work because Impala 1.3 is not yet released for CDH 4.

If installing CDH 4 packages, the Impala 1.3.0 option listed in the install wizard does not work because Impala 1.3.0 is not yet released for CDH 4.

Workaround: Install using parcels (where the unreleased version of Impala does not appear), or select a different version of Impala when installing with packages.

When updating dynamic resource pools, Cloudera Manager updates roles but may fail to update role information displayed in the UI

When updating dynamic resource pools, Cloudera Manager automatically refreshes the affected roles, but they sometimes get marked incorrectly as running with outdated configurations and requiring a refresh.

Workaround: Invoke the Refresh Cluster command from the cluster actions drop-down menu.

User should be prompted to add the AMON role when adding MapReduce to a CDH 5 cluster

When the MapReduce service is added to a CDH 5 cluster, the user is not asked to add the AMON role. Then, an error displays when the user tries to view MapReduce activities.

Workaround: Manually add the AMON role after adding the MapReduce service.

Enterprise license expiration alert not displayed until Cloudera Manager Server is restarted

When an enterprise license expires, the expiration notification banner is not displayed until the Cloudera Manager Server has been restarted (although the expired enterprise features stop working immediately upon license expiration, as expected).

Workaround: None.

Cluster installation with CDH 4.1 and Impala fails

In Cloudera Manager 5.0, installing a new cluster through the wizard with CDH 4.1 and Impala fails with the following error message, "dfs.client.use.legacy.blockreader.local is not enabled."

Workaround: Perform one of the following:
  1. Use CDH 4.2 or higher, or
  2. Install all desired services except Impala in your initial cluster setup. From the home page, use the drop-down menu near the cluster name and select Configure CDH Version. Confirm the version, then add Impala.

HDFS replication does not work from CDH 5 to CDH 4 with different realms

HDFS replication does not work from CDH 5 to CDH 4 with different realms. This is because authentication fails for services in a non-default realm via the WebHdfs API due to a JDK bug. This has been fixed in JDK6-u34 (b03)) and in JDK7.

Workaround: Use JDK 7 or upgrade JDK6 to at least version u34.

Upgrade from Cloudera Manager 5.0.0 beta 1 or beta 2 to Cloudera Manager 5.0.0 requires assistance from Cloudera Support

Contact Cloudera Support before upgrading from Cloudera Manager 5.0.0 beta 1 or beta 2 to Cloudera Manager 5.0.0.

Workaround: Contact Cloudera Support.

Failure of HDFS Replication between clusters with YARN

HDFS replication between clusters in different Kerberos realms fails when using YARN if the target cluster is CDH 5.

Workaround: Use MapReduce (MRv1) instead of YARN.

Empty box appears over the list of results after adding a tag to a file

When tags are added to an entity, in some cases a white box remains after pressing Enter.

Workaround: Refresh the page to remove the artifact.

Upgrade of secure cluster requires installation of JCE policy files

When upgrading a secure cluster via Cloudera Manager, the upgrade initially fails due to the JDK not having Java Cryptography Extension (JCE) unlimited strength policy files. This is because Cloudera Manager installs a copy of the Java 7 JDK during the upgrade, which does not include the unlimited strength policy files. To ensure that unlimited strength functionality continues to work, install the unlimited strength JCE policy files immediately after completing the Cloudera Manager Upgrade Wizard and before taking any other actions in Cloudera Manager.

Workaround: Install the unlimited strength JCE policy files immediately after completing the Cloudera Manager Upgrade Wizard and before taking any other action in Cloudera Manager.

The Details page for MapReduce jobs displays the wrong id for YARN-based replications

The Details link for MapReduce jobs is wrong for YARN-based replications.

Workaround: Find the job id in the link and then go to the YARN Applications page and look for the job there.

The Spark Upload Jar command fails in a secure cluster

The Spark Upload Jar command fails in a secure cluster.

Workaround: To run Spark on YARN, manually upload the Spark assembly jar to HDFS /user/spark/share/lib. The Spark assembly jar is located on the local filesystem, typically in /usr/lib/spark/assembly/lib or /opt/cloudera/parcels/CDH/lib/spark/assembly/lib.

Configurations for decommissioned roles not migrated from MapReduce to YARN

When the Import MapReduce Configuration wizard is used to import MapReduce configurations to YARN, decommissioned roles in the MapReduce service do not cause the corresponding imported roles to be marked as decommissioned in YARN.

Workaround: Delete or decommission the roles in YARN after running the import.

The HDFS command Roll Edits does not work in the UI when HDFS is federated

The HDFS command Roll Edits does not work in the Cloudera Manager UI when HDFS is federated because the command doesn't know which nameservice to use.

Workaround: Use the API, not the Cloudera Manager UI, to execute the Roll Edits command.

Cloudera Manager reports a confusing version number if you have oozie-client, but not oozie installed on a CDH 4.4 node

In CDH versions before 4.4, the metadata identifying Oozie was placed in the client, rather than the server package. Consequently, if the client package is not installed, but the server is, Cloudera Manager will report Oozie has been present but as coming from CDH 3 instead of CDH 4.

Workaround: Either install the oozie-client package, or upgrade to at least CDH 4.4. Parcel based installations are unaffected.

The Sqoop Upgrade command in Cloudera Manager may report success even when the upgrade fails

Workaround: Do one of the following:
    1. Click the Sqoop service and then the Instances tab.
    2. Click the Sqoop server role then the Commands tab.
    3. Click the stdout link and scan for the Sqoop Upgrade command.
  • In the All Recent Commands page, select the stdout link for latest Sqoop Upgrade command.
Verify that the upgrade did not fail.

Cloudera Manager doesn't work with CDH 5.0.0 Beta 1

When you upgrade from Cloudera Manager 5.0.0 Beta 1 with CDH 5.0.0 Beta 1 to Cloudera Manager 5.0.0 Beta 2, Cloudera Manager won't work with CDH 5.0.0 Beta 1 and there's no notification of that fact.

Workaround: None. Do a new installation of CDH 5.0.0 Beta 2.

On CDH 4.1 secure clusters managed by Cloudera Manager 4.8.1 and higher, the Impala Catalog server needs advanced configuration snippet update

Impala queries fail on CDH 4.1 when Hive "Bypass Hive Metastore Server" option is selected.

Workaround: Add the following to Impala catalog server advanced configuration snippet for hive-site.xml, replacing Hive_Metastore_Server_Host with the host name of your Hive Metastore Server:

<property> 
<name>hive.metastore.local</name> 
<value>false</value> 
</property> 
<property> 
<name>hive.metastore.uris</name> 
<value>thrift://Hive_Metastore_Server_Host:9083</value>
</property>

Rolling Upgrade to CDH 5 is not supported.

Rolling upgrade between CDH 4 and CDH 5 is not supported. Incompatibilities between major versions means rolling restarts are not possible. In addition, rolling upgrade will not be supported from CDH 5.0.0 Beta 1 to any later releases, and may not be supported between any future beta versions of CDH 5 and the General Availability release of CDH 5.

Workaround: None.

Error reading .zip file created with the Collect Diagnostic Data command.

After collecting Diagnostic Data and using the Download Diagnostic Data button to download the created zip file to the local system, the zip file cannot be opened using the FireFox browser on a Macintosh. This is because the zip file is created as a Zip64 file, and the unzip utility included with Macs does not support Zip64. The zip utility must be version 6.0 or later. You can determine the zip version with unzip -v.

Workaround: Update the unzip utility to a version that supports Zip64.

Enabling wildcarding in a secure environment causes NameNode to fail to start.

In a secure cluster, you cannot use a wildcard for the NameNode's RPC or HTTP bind address, or the NameNode will fail to start. For example, dfs.namenode.http-address must be a real, routable address and port, not 0.0.0.0.port. In Cloudera Manager, the "Bind NameNode to Wildcard Address" property must not be enabled. This should affect you only if you are running a secure cluster and your NameNode needs to bind to multiple local addresses.

Bug: HDFS-4448

Severity: Medium

Workaround: Disable the "Bind NameNode to Wildcard Address" property found under the Configuration tab for the NameNode role group.

After JobTracker failover, complete jobs from the previous active JobTracker are not visible.

When a JobTracker failover occurs and a new JobTracker becomes active, the new JobTracker UI does not show the completed jobs from the previously active JobTracker (that is now the standby JobTracker). For these jobs the "Job Details" link does not work.

Severity: Med

Workaround: None.

After JobTracker failover, information about rerun jobs is not updated in Activity Monitor.

When a JobTracker failover occurs while there are running jobs, jobs are restarted by the new active JobTracker by default. For the restarted jobs the Activity Monitor will not update the following: 1) The start time of the restarted job will remain the start time of the original job. 2) Any Map or Reduce task that had finished before the failure happened will not be updated with information about the corresponding task that was rerun by the new active JobTracker.

Severity: Med

Workaround: None.

Installing on AWS, you must use private EC2 hostnames.

When installing on an AWS instance, and adding hosts using their public names, the installation will fail when the hosts fail to heartbeat.

Severity: Med

Workaround:

Use the Back button in the wizard to return to the original screen, where it prompts for a license.

Rerun the wizard, but choose "Use existing hosts" instead of searching for hosts. Now those hosts show up with their internal EC2 names.

Continue through the wizard and the installation should succeed.

After removing and then re-adding a service, the alternatives settings are incorrect.

After deleting a service, the alternatives settings are not cleaned up. If you then re-add the service, it will be given a new instance name, and a new set of configurations settings are added. However, because both the new and old (deleted) instances have the same alternatives priority, the original one will be used rather than the newer one.

Severity: Med

Workaround: The simplest way to fix this is:

  1. Go to the Configuration tab for the new service instance in Cloudera Manager
  2. Search for "alternatives".
  3. Raise the priority value and save the setting.
  4. Redeploy the client configuration.

If HDFS uses Quorum-based Storage without HA enabled, the SecondaryNameNode cannot checkpoint.

If HDFS is set up in non-HA mode, but with Quorum-based storage configured, the dfs.namenode.edits.dir is automatically configured to the Quorum-based Storage URI. However, the SecondaryNameNode cannot currently read the edits from a Quorum-based Storage URI, and will be unable to do a checkpoint.

Severity: Medium

Workaround: Add to the NameNode's advanced configuration snippet the dfs.namenode.edits.dir property with both the value of the Quorum-based Storage URI as well as a local directory, and restart the NameNode. For example,

<property> <name>dfs.namenode.edits.dir</name>
<value>qjournal://jn1HostName:8485;jn2HostName:8485;jn3HostName:8485/journalhdfs1,file:///dfs/edits</value>
</property>

Changing the rack configuration may temporarily cause mis-replicated blocks to be reported.

A rack re-configuration will cause HDFS to report mis-replicated blocks until HDFS rebalances the system, which may take some time. This is a normal side-effect of changing the configuration.

Severity: Low

Workaround: None

Cannot use '/' as a mount point with a Federated HDFS Nameservice.

A Federated HDFS Service doesn't support nested mount points, so it is impossible to mount anything at '/'. Because of this issue, the root directory will always be read-only, and any client application that requires a writeable root directory will fail.

Severity: Low

Workaround:
  1. In the CDH 4 HDFS Service > Configuration tab of the Cloudera Manager Admin Console, search for "nameservice".
  2. In the Mountpoints field, change the mount point from "/" to a list of mount points that are in the namespace that the Nameservice will manage. (You can enter this as a comma-separated list - for example, "/hbase, /tmp, /user" or by clicking the plus icon to add each mount point in its own field.) You can determine the list of mount points by running the command hadoop fs -ls / from the CLI on the NameNode host.

Historical disk usage reports do not work with federated HDFS.

Severity: Low

Workaround: None.

(CDH 4 only) Activity monitoring does not work on YARN activities.

Activity monitoring is not supported for YARN in CDH 4.

Severity: Low

Workaround: None

HDFS monitoring configuration applies to all Nameservices

The monitoring configurations at the HDFS level apply to all Nameservices. So, if there are two federated Nameservices, it's not possible to disable a check on one but not the other. Likewise, it's not possible to have different thresholds for the two Nameservices.

Severity: Low

Workaround: None

Supported and Unsupported Replication Scenarios and Limitations

Restoring snapshot of a file to an empty directory does not overwrite the directory

Restoring the snapshot of an HDFS file to an HDFS path that is an empty HDFS directory (using the Restore As action) will result in the restored file present inside the HDFS directory instead of overwriting the empty HDFS directory.

Workaround: None.

HDFS Snapshot appears to fail if policy specifies duplicate directories.

In an HDFS snapshot policy, if a directory is specified more than once, the snapshot appears to fail with an error message on the Snapshot page. However, in the HDFS Browser, the snapshot is shown as having been created successfully.

Severity: Low

Workaround: Remove the duplicate directory specification from the policy.

Hive replication fails if "Force Overwrite" is not set.

The Force Overwrite option, if checked, forces overwriting data in the target metastore if there are incompatible changes detected. For example, if the target metastore was modified and a new partition was added to a table, this option would force deletion of that partition, overwriting the table with the version found on the source. If the Force Overwrite option is not set, recurring replications may fail.

Severity: Med

Workaround: Set the Force Overwrite option.

During HDFS replication, tasks may fail due to DataNode timeouts.

In CDH 4.2, during an HDFS replication job (using Cloudera Manager's Backup and Data Recovery product) individual tasks in the Replication job may fail due to DataNode timeouts. If enough of these timeouts occur, the replication task may slow down, and the entire replication job could time out and fail.

Severity: Medium

Workaround: None.

Cloudera Manager 5.1.x System Requirements:

Supported Operating Systems

Supported JDK Versions

Supported Databases

Supported Supported CDH and Other Managed Service Versions

Resource Requirements

Networking and Security Requirements

Cloudera Manager Quick Start Guide

This quick start guide describes how to quickly create a new installation of Cloudera Manager 5, CDH 5, and managed services on a cluster of four hosts. The resulting deployment can be used for demonstrations and proof of concept applications, but is not recommended for production.


Requirements

The four hosts in the cluster must satisfy the following requirements:
  • The hosts must have at least 10 GB RAM
  • You must have root or password-less sudo access to the hosts
  • If using root, the hosts must accept the same root password
  • The hosts must have Internet access to allow the wizard to install software from archive.cloudera.com
  • Run a supported OS:
    • RHEL-compatible systems
      • Red Hat Enterprise Linux and CentOS 5.7, 64-bit
      • Red Hat Enterprise Linux and CentOS 6.2, 64-bit
      • Red Hat Enterprise Linux and CentOS 6.4, 64-bit
      • Oracle Enterprise Linux 6.4, 64-bit
      • Oracle Enterprise Linux 5.6, 64-bit
    • SLES - SUSE Linux Enterprise Server 11, 64-bit. Service Pack 2 or later is required. The Updates repository must be active and SUSE Linux Enterprise Software Development Kit 11 SP1 is required.
    • Debian - Debian 7.0 and 7.1, 64-bit
    • Ubuntu - Ubuntu 12.04, 64-bit
If your environment does not satisfy these requirements, the procedure described in this guide may not be appropriate for you. For information about other Cloudera Manager installation options and requirements, see the Cloudera Manager Installation Guide.

Download and Run the Cloudera Manager Server Installer

  1. Download the Cloudera Manager installer binary from Cloudera Manager Downloads to the cluster host where you want to install the Cloudera Manager Server.
    1. Under Cloudera Manager 5.x downloads, click Download Cloudera Manager Express Edition.
    2. Optionally register and click Submit or click the Just take me to the download page link.
    The cloudera-manager-installer.bin file downloads.
  2. Change cloudera-manager-installer.bin to have executable permission.
    $ chmod u+x cloudera-manager-installer.bin
  3. Run the Cloudera Manager Server installer.
    $ sudo ./cloudera-manager-installer.bin
  4. Read the Cloudera Manager README and then press Return or Enter to choose Next.
  5. Read the Cloudera Manager Standard License and then press Return or Enter to choose Next. Use the arrow keys and press Return or Enter to choose Yes to confirm you accept the license.
  6. Read the Oracle Binary Code License Agreement and then press Return or Enter to choose Next.
  7. When the installation completes, the complete URL provided for the Cloudera Manager Admin Console, including the port number, which is 7180 by default. Press Return or Enter to choose OK to continue.
  8. Press Return or Enter to choose OK to exit the installer.
  9. On RHEL 5 and CentOS 5, install Python 2.6 or 2.7. Download the appropriate repository rpm packages to the Cloudera Manager Server host and then install Python using yum. For example, use the following commands:
    $ su -c 'rpm -Uvh http://download.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm'
    ...
    $ yum install python26
  Note: If the installation is interrupted for some reason, you may need to clean up before you can re-run it. See Uninstalling Cloudera Manager and Managed Software in the Cloudera Manager Installation Guide.

Start the Cloudera Manager Admin Console

  1. Wait several minutes for the Cloudera Manager Server to complete its startup. To observe the startup process you can perform tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log on the Cloudera Manager Server host. If the Cloudera Manager Server does not start, see Troubleshooting Installation and Upgrade Problems.
  2. In a web browser, enter http://Server host:7180, where Server host is the fully-qualified domain name or IP address of the host where you installed the Cloudera Manager Server. The login screen for Cloudera Manager Admin Console displays.
  3. Log into Cloudera Manager Admin Console with the credentials: Username: admin Password: admin.

Install and Configure Software Using the Cloudera Manager Wizard

Installing and configuring Cloudera Manager, CDH, and managed service software on the cluster hosts involves the following three main steps.

Choose Cloudera Manager Edition and Specify Hosts

  1. Choose Cloudera Enterprise Data Hub Edition Trial, which does not require a license, but expires after 60 days and cannot be renewed. The trial allows you to create all CDH and managed services supported by Cloudera Manager. Click Continue.
  2. Information is displayed indicating what edition of Cloudera Manager will be installed and the services you can choose from. Click Continue.
  3. Specify the hosts on which to install CDH and managed services. You can specify hostnames and/or IP addresses and ranges, for example: 10.1.1.[1-4] or host[1-3].company.com. You can specify multiple addresses and address ranges by separating them by commas, semicolons, tabs, or blank spaces, or by placing them on separate lines.
  4. Click Search. Cloudera Manager identifies the hosts on your cluster. Verify that the number of hosts shown matches the number of hosts where you want to install services. Deselect host entries that do not exist and deselect the hosts where you do not want to install services. Click Continue. The Select Repository page displays.

Install CDH and Managed Service Software

  1. Keep the default distribution method Use Parcels and the default version of CDH 5.
  2. For the Cloudera Manager Agent, keep the default Matched release for this Cloudera Manager Server. Click Continue twice.
  3. Specify host SSH login properties:
    1. Keep the default login root or enter the user name for an account that has password-less sudo permission.
    2. If you choose to use password authentication, enter and confirm the password.
    Click Continue. Cloudera Manager installs the Oracle JDK and the Cloudera Manager Agent packages on each host and starts the Agent.
  4. Click Continue. Cloudera Manager installs CDH. During the parcel installation, progress is indicated for the two phases of the parcel installation process (Download and Distribution) in a separate progress bars. When the Continue button appears at the bottom of the screen, the installation process is completed.
  5. Click Continue. The Host Inspector runs to validate the installation, and provides a summary of what it finds, including all the versions of the installed components. If the validation is successful, click Finish. The Cluster Setup page displays.

Add and Configure Services

  1. Click the All Services radio button to create HDFS, YARN (includes MapReduce 2), ZooKeeper, Oozie, Hive, Hue, Sqoop, HBase, Impala, Solr, Spark, and Key-Value Store Indexer services. Click Continue. The Customize Role Assignments page displays.
  2. Configure the following role assignments:
    • Click the text field under the HBase Thrift Server role. In the host selection dialog that displays, check the checkbox next to any host and click OK at the bottom right.
    • Click the text field under the Server role of the ZooKeeper service. In the host selection dialog that displays, uncheck the checkbox next to the host assigned by default (the master host) and check checkboxes next to the remaining three hosts. Click OK at the bottom right.
    Click Continue. The Database Setup page displays.
  3. Leave the default setting of Use Embedded Database to have Cloudera Manager create and configure all required databases in an embedded PostgreSQL database. Click Test Connection. When the test completes with a , click Continue. The Review Changes page displays.
  4. Review the configuration changes to be applied. Click Continue. The Command Progress page displays.
  5. The wizard performs 32 steps to configure and starts the services. When the startup completes, click Continue.
  6. A success message displays indicating that the cluster has been successfully started. Click Finish to proceed to the Home page.

Test the Installation

The Home page should look something like this:

On the left side of the screen is a list of services currently running with their status information. All the services should be running with Good Health , however there may be a small number of configuration warnings indicated by a wrench icon and a numbered badge , which you can ignore.

You can click each service to view more detailed information about the service. You can also test your installation by running a MapReduce job or interacting with the cluster with a Hue application.

Running a MapReduce Job

  1. Log into a cluster host.
  2. Run the Hadoop PiEstimator example:
    sudo -u hdfs hadoop jar \ 
    /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar \
    pi 10 100
  3. View the result of running the job by selecting the following from the top navigation bar in the Cloudera Manager Admin Console: Clusters > Cluster 1 > YARN (MR2 Included) Applications. You will see an entry like the following:

Testing with Hue

A good way to test the cluster is by running a job. In addition, you can test the cluster by running one of the Hue web applications. Hue is a graphical user interface that allows you to interact with your clusters by running applications that let you browse HDFS, manage a Hive metastore, and run Hive, Impala, and Search queries, Pig scripts, and Oozie workflows.
  1. In the Cloudera Manager Admin Console Home page, click the Hue service.
  2. Click the Hue Web UI tab, which opens Hue in a new window.
  3. Log in with the credentials, username: hdfs, password: hdfs.
  4. Choose an application in the navigation bar at the top of the browser window.

For more information, see the Hue User Guide.