Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×

Sign in or complete our product interest form to continue.

Please Read and Accept our Terms


The recommended tool for installing Cloudera Enterprise

This download installs Cloudera Enterprise or Cloudera Express.

 

Cloudera Enterprise requires a license; however, when installing Cloudera Express you will have the option to unlock Cloudera Enterprise features for a free 60-day trial.

 

Once the trial has concluded, the Cloudera Enterprise features will be disabled until you obtain and upload a license.

 

Note: All CDH and Cloudera Manager hosts that make up a logical cluster need to run on the same major OS release to be covered by Cloudera Support.

For a table of supported operating systems, see CDH and Cloudera Manager 5.8.x Supported Operating Systems

 

  • Important: Cloudera supports RHEL 7 with the following limitations:
  • SLES - SUSE Linux Enterprise Server 11, Service Pack 4, 64-bit is supported by CDH 5.7 and higher. Service Packs 2 and 3 are supported by CDH 5.0 through CDH 5.6. Service Pack 1 is not supported by CDH 5, only by CDH 4. Hosts running Cloudera Manager Agents must use SUSE Linux Enterprise Software Development Kit 11 SP1.
  • Debian - Jessie 8.2, Wheezy 7.0, 7.1, and 7.8, 64-bit. (Squeeze 6.0 is only supported by CDH 4.)
  • Ubuntu - Trusty 14.04 (LTS) and Precise 12.04 (LTS), 64-bit. (Lucid 10.04 is only supported by CDH 4.)

Note:

  • Cloudera Enterprise is supported on platforms with Security-Enhanced Linux (SELinux) enabled. Cloudera is not responsible for policy support nor policy enforcement. If you experience issues with SELinux, contact your OS provider.
  • CDH 5.8 DataNode hosts with EMC® DSSD™ D5™ are supported by RHEL 6.6, 7.1, and 7.2.
Selected tab: SupportedOperatingSystems

The version of Oracle JDK supported by Cloudera Manager depends on the version of CDH being managed.For more information see CDH and Cloudera Manager Supported JDK Versions.

 

Cloudera Manager can install Oracle JDK 1.7.0_67 during installation and upgrade. If you prefer to install the JDK yourself, follow the instructions in Java Development Kit Installation.

Selected tab: SupportedJDKVersions

The Cloudera Manager Admin Console, which you use to install, configure, manage, and monitor services, supports the following browsers:

  • Mozilla Firefox 24 and 31.
  • Google Chrome 36 and higher.
  • Internet Explorer 9 and higher. Internet Explorer 11 Native Mode.
  • Safari 5 and higher.
Selected tab: SupportedBrowsers

For a list of databases supported by Cloudera Manager, see CDH and Cloudera Manager Supported Databases.

Cloudera Manager requires several databases. The Cloudera Manager Server stores information about configured services, role assignments, configuration history, commands, users, and running processes in a database of its own. You must also specify a database for the Activity Monitor and Reports Manager roles.

Important: When processes restart, the configuration for each of the services is redeployed using information that is saved in the Cloudera Manager database. If this information is not available, your cluster will not start or function correctly. You must therefore schedule and maintain regular backups of the Cloudera Manager database to recover the cluster in the event of the loss of this database.

The database you use must be configured to support UTF8 character set encoding. The embedded PostgreSQL database that is installed when you follow Installation Path A - Automated Installation by Cloudera Manager (Non-Production Mode) automatically provides UTF8 encoding. If you install a custom database, you may need to enable UTF8 encoding. The commands for enabling UTF8 encoding are described in each database topic under Cloudera Manager and Managed Service Datastores.

After installing a database, upgrade to the latest patch version and apply any other appropriate updates. Available updates may be specific to the operating system on which it is installed.

Cloudera supports the shipped version of MariaDB, MySQL and PostgreSQL for each supported Linux distribution. Each database is supported for all components in Cloudera Manager and CDH subject to the notes in CDH 4 Supported Databases and CDH 5 Supported Databases.

Selected tab: SupportedDatabases

The following versions of CDH and managed services are supported:Warning: Cloudera Manager 5 does not support CDH 3 and you cannot upgrade Cloudera Manager 4 to Cloudera Manager 5 if you have a cluster running CDH 3. Therefore, to upgrade CDH 3 clusters to CDH 4 using Cloudera Manager, you must use Cloudera Manager 4.

  • CDH 4 and CDH 5. The latest released versions of CDH 4 and CDH 5 are strongly recommended. For information on CDH 4 requirements, see CDH 4 Requirements and Supported Versions. For information on CDH 5 requirements, see CDH 5 Requirements and Supported Versions.
  • Cloudera Impala - Cloudera Impala is included with CDH 5. Cloudera Impala 1.2.1 with CDH 4.1.0 or higher. For more information on Impala requirements with CDH 4, see Impala Requirements.
  • Cloudera Search - Cloudera Search is included with CDH 5. Cloudera Search 1.2.0 with CDH 4.6.0. For more information on Cloudera Search requirements with CDH 4, see Cloudera Search Requirements.
  • Apache Spark - 0.90 or higher with CDH 4.4.0 or higher.
  • Apache Accumulo - 1.4.3 with CDH 4.3.0, 1.4.4 with CDH 4.5.0, and 1.6.0 with CDH 4.6.0.

For more information, see the Product Compatibility Matrix.

Selected tab: SupportedCDHandManagedServiceVersions

See CDH and Cloudera Manager Supported Transport Layer Security Versions.

 

To configure TLS security for the Cloudera Manager Server and Agents, see Configuring TLS Security for Cloudera Manager.

Selected tab: SupportedTransportLayerSecurityVersions

Cloudera Manager requires the following resources:

  • Disk Space
    • Cloudera Manager Server
      • 5 GB on the partition hosting /var.
      • 500 MB on the partition hosting /usr.
      • For parcels, the space required depends on the number of parcels you download to the Cloudera Manager Server and distribute to Agent hosts. You can download multiple parcels of the same product, of different versions and different builds. If you are managing multiple clusters, only one parcel of a product/version/build/distribution is downloaded on the Cloudera Manager Server—not one per cluster. In the local parcel repository on the Cloudera Manager Server, the approximate sizes of the various parcels are as follows:
        • CDH 5 (which includes Impala and Search) - 1.5 GB per parcel (packed), 2 GB per parcel (unpacked)
        • Impala - 200 MB per parcel
        • Cloudera Search - 400 MB per parcel
    • Cloudera Management Service -The Host Monitor and Service Monitor databases are stored on the partition hosting /var. Ensure that you have at least 20 GB available on this partition.
    • Agents - On Agent hosts, each unpacked parcel requires about three times the space of the downloaded parcel on the Cloudera Manager Server. By default, unpacked parcels are located in/opt/cloudera/parcels.
  • RAM - 4 GB is recommended for most cases and is required when using Oracle databases. 2 GB might be sufficient for non-Oracle deployments with fewer than 100 hosts. However, to run the Cloudera Manager Server on a machine with 2 GB of RAM, you must tune down its maximum heap size (by modifying -Xmx in /etc/default/cloudera-scm-server). Otherwise the kernel might kill the Server for consuming too much RAM.
  • Python - Cloudera Manager requires Python 2.4 or higher, but Hue in CDH 5 and package installs of CDH 5 require Python 2.6 or 2.7. All supported operating systems include Python version 2.4 or higher.
  • Perl - Cloudera Manager requires perl.
Selected tab: ResourceRequirements

The hosts in a Cloudera Manager deployment must satisfy the following networking and security requirements:

  • CDH requires IPv4. IPv6 is not supported and must be disabled.

    See also

  • Multihoming CDH or Cloudera Manager is not supported outside specifically certified Cloudera partner appliances. Cloudera finds that current Hadoop architectures combined with modern network infrastructures and security practices remove the need for multihoming. Multihoming, however, is beneficial internally in appliance form factors to take advantage of high-bandwidth InfiniBand interconnects.
  • Although some subareas of the product might work with unsupported custom multihoming configurations, there are known issues with multihoming. In addition, unknown issues can arise because multihoming is not covered by the test matrix outside the Cloudera-certified partner appliances.
  • Cluster hosts must have a working network name resolution system and correctly formatted/etc/hosts file. All cluster hosts must have properly configured forward and reverse host resolution through DNS. The /etc/hosts files must:
    • Contain consistent information about hostnames and IP addresses across all hosts
    • Not contain uppercase hostnames
    • Not contain duplicate IP addresses

    Cluster hosts must not use aliases, either in /etc/hosts or in configuring DNS. A properly formatted/etc/hosts file should be similar to the following example:

     

    127.0.0.1
    localhost.localdomain
    localhost
    192.168.1.1
    cluster-01.example.com
    cluster-01
    192.168.1.2
    cluster-02.example.com
    cluster-02
    192.168.1.3
    cluster-03.example.com
    cluster-03

     

  • In most cases, the Cloudera Manager Server must have SSH access to the cluster hosts when you run the installation or upgrade wizard. You must log in using a root account or an account that has password-less sudo permission. For authentication during the installation and upgrade procedures, you must either enter the password or upload a public and private key pair for the root or sudo user account. If you want to use a public and private key pair, the public key must be installed on the cluster hosts before you use Cloudera Manager.

    Cloudera Manager uses SSH only during the initial install or upgrade. Once the cluster is set up, you can disable root SSH access or change the root password. Cloudera Manager does not save SSH credentials, and all credential information is discarded when the installation is complete.

  • If single user mode is not enabled, the Cloudera Manager Agent runs as root so that it can make sure the required directories are created and that processes and files are owned by the appropriate user (for example, the hdfs and mapred users).
  • No blocking is done by Security-Enhanced Linux (SELinux).Note: Cloudera Enterprise is supported on platforms with Security-Enhanced Linux (SELinux) enabled. Cloudera is not responsible for policy support nor policy enforcement. If you experience issues with SELinux, contact your OS provider.
  • No blocking by iptables or firewalls; port 7180 must be open because it is used to access Cloudera Manager after installation. Cloudera Manager communicates using specific ports, which must be open.
  • For RHEL and CentOS, the /etc/sysconfig/network file on each host must contain the hostname you have just set (or verified) for that host.
  • Cloudera Manager and CDH use several user accounts and groups to complete their tasks. The set of user accounts and groups varies according to the components you choose to install. Do not delete these accounts or groups and do not modify their permissions and rights. Ensure that no existing systems prevent these accounts and groups from functioning. For example, if you have scripts that delete user accounts not in a whitelist, add these accounts to the list of permitted accounts. Cloudera Manager, CDH, and managed services create and use the following accounts and groups:

Users and Groups

Component (Version)

Unix User ID Groups Notes
Cloudera Manager (all versions) cloudera-scm cloudera-scm Cloudera Manager processes such as the Cloudera Manager Server and the monitoring roles run as this user.

The Cloudera Manager keytab file must be named cmf.keytab since that name is hard-coded in Cloudera Manager.Note: Applicable to clusters managed by Cloudera Manager only.

Apache Accumulo (Accumulo 1.4.3 and higher) accumulo accumulo Accumulo processes run as this user.
Apache Avro   No special users.
Apache Flume (CDH 4, CDH 5) flume flume The sink that writes to HDFS as this user must have write privileges.
Apache HBase (CDH 4, CDH 5) hbase hbase The Master and the RegionServer processes run as this user.
HDFS (CDH 4, CDH 5) hdfs hdfs, hadoop The NameNode and DataNodes run as this user, and the HDFS root directory as well as the directories used for edit logs should be owned by it.
Apache Hive (CDH 4, CDH 5) hive hive

The HiveServer2 process and the Hive Metastore processes run as this user.

A user must be defined for Hive access to its Metastore DB (for example, MySQL or Postgres) but it can be any identifier and does not correspond to a Unix uid. This isjavax.jdo.option.ConnectionUserName in hive-site.xml.

Apache HCatalog (CDH 4.2 and higher, CDH 5) hive hive

The WebHCat service (for REST access to Hive functionality) runs as the hive user.

HttpFS (CDH 4, CDH 5) httpfs httpfs

The HttpFS service runs as this user. See HttpFS Security Configuration for instructions on how to generate the merged httpfs-http.keytab file.

Hue (CDH 4, CDH 5) hue hue

Hue services run as this user.

Cloudera Impala (CDH 4.1 and higher, CDH 5) impala impala, hive Impala services run as this user.
Apache Kafka (Cloudera Distribution of Kafka 1.2.0) kafka kafka Kafka services run as this user.
Java KeyStore KMS (CDH 5.2.1 and higher) kms kms The Java KeyStore KMS service runs as this user.
Key Trustee KMS (CDH 5.3 and higher) kms kms The Key Trustee KMS service runs as this user.
Key Trustee Server (CDH 5.4 and higher) keytrustee keytrustee The Key Trustee Server service runs as this user.
Kudu kudu kudu Kudu services run as this user.
Llama (CDH 5) llama llama Llama runs as this user.
Apache Mahout   No special users.
MapReduce (CDH 4, CDH 5) mapred mapred, hadoop Without Kerberos, the JobTracker and tasks run as this user. The LinuxTaskController binary is owned by this user for Kerberos.
Apache Oozie (CDH 4, CDH 5) oozie oozie The Oozie service runs as this user.
Parquet   No special users.
Apache Pig   No special users.
Cloudera Search (CDH 4.3 and higher, CDH 5) solr solr The Solr processes run as this user.
Apache Spark (CDH 5) spark spark The Spark History Server process runs as this user.
Apache Sentry (CDH 5.1 and higher) sentry sentry The Sentry service runs as this user.
Apache Sqoop (CDH 4, CDH 5) sqoop sqoop This user is only for the Sqoop1 Metastore, a configuration option that is not recommended.
Apache Sqoop2 (CDH 4.2 and higher, CDH 5) sqoop2 sqoop, sqoop2 The Sqoop2 service runs as this user.
Apache Whirr   No special users.
YARN (CDH 4, CDH 5) yarn yarn, hadoop Without Kerberos, all YARN services and applications run as this user. The LinuxContainerExecutor binary is owned by this user for Kerberos.
Apache ZooKeeper (CDH 4, CDH 5) zookeeper zookeeper The ZooKeeper processes run as this user. It is not configurable.
Selected tab: NetworkingandSecurityRequirements
Selected tab: SystemRequirements

Issues Fixed in Cloudera Manager 5.8.2

 

Improve advanced configuration snippet redaction to encompass cloud provider credentials and other access tokens

 

The redaction of potentially sensitive parameters in advanced configuration snippets is extended to those commonly used by the Azure Data Lake.

 

 

If files excluded by exclusion filters are renamed, they are not replicated

 

Customers using Incremental HDFS replication had an issue where if an excluded file (through exclusion filters) is renamed to an 'included' file, the new file is still not copied to the destination cluster. This issue is resolved as part of this fix.

 

 

Hive Replication Command should update copy the Serde properties correctly

 

After this fix, Hive Replication replicates the Serde Properties and also copies corresponding HDFS file.

 

 

Increase default Solrd watchdog Timeout value

 

Solr server initialization can take up to 60 secs to complete. During this time interval, Solr server does not respond to the solrd watchdog requests. This can result in solrd watchdog terminating the Solr server process. The default timeout duration for watchdog is increased to 70 secs.

 

 

Add service changes YARN settings

 

In Cloudera Manager 5.7, adding any new service to your cluster can cause the YARN setting for mapreduce.job.reduces to change unexpectedly. Adding a service no longer causes this problem.

 

 

OPSAPS-29327 Add config for hive.metastore.server.max.message.size

 

Hive Metastore max message size can be configured now using Max Message Size for Hive MetaStore. It defaults to 100MB. It can cause staleness for customers on Cloudera Manager upgrade.

 

 

XSS in Kerberos activation

 

In lower releases, there was an XSS vulnerability on the Kerberos page. This is now fixed.

 

 

Upload deployment.json fails when it contains replication info

 

While trying to migrate Cloudera Manager using deployment.json file with existing Hive replication, schedules used to fail. This has been fixed in this release.

 

 

Cloudera Manager set cataloged default jvm memory to 4G may cause oom on upgrade to Cloudera Manager 5.7+

 

After upgrading to 5.7 or later, customers could see a reduced Java heap maximum on Impala Catalog Server, due to a change in its default value. Upgrading from Cloudera Manager < 5.7 to Cloudera Manager 5.8.2 no longer sees any effective change in the Impala Catalog Server Java Heap size.

 

 

Oozie first run fails with custom principals

 

In 5.8.1, Oozie first run fails if using kerberos custom principals. This is now fixed.

 

 

Hive Replication shows "Dry Run" incorrectly

 

The earlier known issue that running Hive Replication shows "Dry Run" in status message is fixed now.

 

 

HDFS Snapshot policy is selecting unhealthy host to run on

 

Policy for selecting a role to run HDFS Snapshot command: We select a non-decommissioned host that is in Active status. Also, hosts in maintenance mode have a lower priority than hosts in active state.

 

 

Cluster export fails when service configuration is invalid

 

The export cluster template code path was failing because of stale configuration in the Cloudera Manager database. Having a stale configuration in the database is possible. This could happen when configurations are deprecated on older CDH releases.

 

 

Agent orphan cleanup removes process dir from in flight process

 

With this fix, the preventative steps in TSB-181 are no longer required.

 

 

Fix CatalogServiceClient to handle TLS connections to catalogd for UDF replication

 

When Impala uses SSL, Cloudera supports TLS Connection to Catalog Server. Customers are able to enable replication for any Impala UDFs/Metadata (in Hive Replication).

 

 

Redact Content from Flume config

 

Enabled redacting sensitive information from Flume configuration.

 

 

Host inspector incorrectly warns about kernel version "2.6.32-504.16.2"

 

Host inspector incorrectly warns about kernel version "2.6.32-504.16.2" as "non-recommended."

 

 

Impala Breakpad script does not convert exponentials into decimals and leads to errors

 

Impala Breakpad script failure that happens when trying to collect more than 10 MB of dumps from a single role is fixed.

 

 

Oozie points to older sharelib even after running sharelib install command

 

After an "Install Oozie Share Lib" action, the Oozie service is informed that that there is a new shared lib installed. This eliminates the need for a separate manual restart.

 

Selected tab: WhatsNew

Want to Get Involved or Learn More?

Check out our other resources

Cloudera Community

Collaborate with your peers, industry experts, and Clouderans to make the most of your investment in Hadoop.

Cloudera University

Receive expert Hadoop training through Cloudera University, the industry's only truly dynamic Hadoop training curriculum that’s updated regularly to reflect the state of the art in big data.