Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×

Long term component architecture

As the main curator of open standards in Hadoop, Cloudera has a track record of bringing new open source solutions into its platform (such as Apache Spark, Apache HBase, and Apache Parquet) that are eventually adopted by the community at large. As standards, you can build longterm architecture on these components with confidence.

 

PLEASE NOTE:

With the exception of DSSD support, Cloudera Enterprise 5.6.0 is identical to CDH 5.5.2/Cloudera Manager 5.5.3  If you do not need DSSD support, you do not need to upgrade if you are already using the latest 5.5.x release.

 

Thank you for choosing CDH, your download instructions are below:

Installing the Latest CDH 5 Release

This page explains how to do an unmanaged deployment of CDH 5 from the command line. For a managed deployment, see Cloudera Manager Deployment.

 

CDH 5 Installation Options

There are multiple ways to install CDH 5:Note: Cloudera recommends automatically installing CDH 5 and dependencies with Cloudera Manager.

  • Automatically install CDH 5 with a Cloudera Manager Deployment. This is the simplest and preferred method.
  • Manually install the CDH 5 package or repository in one of three ways:
    • Install the CDH 5 "1-click" package (preferred manual method) OR
    • Add the CDH 5 repository OR
    • Build your own CDH 5 repository.
  • Manually install the CDH 5 tarball. See "Package and Tarball Binaries" below.

 

Package and Tarball Binaries

Installing from Packages

 

Installing from a Tarball

  • The CDH 5 tarball deploys YARN and includes the MRv1 binaries. There is no separate tarball for MRv1. The MRv1 scripts are in the directory, bin-mapreduce1, and examples are in examples-mapreduce1.

 

Before You Begin Installing CDH 5 Manually

Note: Running Services

Use the service command to start, stop, and restart CDH components, rather than running scripts in/etc/init.d directly. The service command creates a predictable environment by setting the current working directory to / and removing most environment variables (passing only LANG and TERM). With/etc/init.d, existing environment variables remain in force and can produce unpredictable results. When you install CDH from packages, service is installed as part of the Linux Standard Base (LSB).

 

Steps to Install CDH 5 Manually

 

Step 1: Add or Build the CDH 5 Repository or Download the "1-click Install" package.

 

  • To install CDH 5 on a RHEL system, download packages with yum or use a web browser.
  • To install CDH 5 on a SLES system, download packages with zypper or YaST or use a web browser.
  • To install CDH 5 on an Ubuntu or Debian system, download packages with apt or use a web browser.

 

On RHEL-compatible Systems

Use one of the following methods to install CDH 5 on RHEL-compatible systems.

Do this on all the systems in the cluster.

To download and install the CDH 5 "1-click Install" package:

  1. Download the CDH 5 "1-click Install" package (or RPM).

    Click the appropriate RPM and Save File to a directory with write access (for example, your home directory).

    OS Version Link to CDH 5 RPM
    RHEL/CentOS/Oracle 5 RHEL/CentOS/Oracle 5 link
    RHEL/CentOS/Oracle 6 RHEL/CentOS/Oracle 6 link
    RHEL/CentOS/Oracle 7 RHEL/CentOS/Oracle 7 link

  2. Install the RPM for all RHEL versions:

    $ sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo yum clean all

 

OR: To add the CDH 5 repository:

Download the repo file. Click the link for your RHEL or CentOS system in the table, find the appropriate repo file, and save in /etc/yum.repos.d/.

For OS Version

Link to CDH 5 Repository

RHEL/CentOS/Oracle 5

RHEL/CentOS/Oracle 5 link

RHEL/CentOS/Oracle 6

RHEL/CentOS/Oracle 6 link

RHEL/CentOS/Oracle 7

RHEL/CentOS/Oracle 7 link

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo yum clean all

 

OR: To build a Yum repository:

Follow the instructions at Creating a Local Yum Repository to create your own yum repository:

  • Download the appropriate repo file
  • Create the repo
  • Distribute the repo and set up a web server.

 

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo yum clean all

 

On SLES Systems

Use one of the following methods to download the CDH 5 repository or package on SLES systems.

To download and install the CDH 5 "1-click Install" package:

  1. Download the CDH 5 "1-click Install" package.

    Download the rpm file, choose Save File, and save it to a directory to which you have write access (for example, your home directory).

  2. Install the RPM:

    $ sudo rpm -i cloudera-cdh-5-0.x86_64.rpm

  3. Update your system package index by running:

    $ sudo zypper refresh

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

OR: To add the CDH 5 repository:

  1. Run the following command:

    $ sudo zypper addrepo -f https://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/cloudera-cdh5.repo

  2. Update your system package index by running:

    $ sudo zypper refresh

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo zypper clean --all

 

OR: To build a SLES repository:

If you want to create your own SLES repository, create a mirror of the CDH SLES directory by followingthese instructions that explain how to create a SLES repository from the mirror.

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo zypper clean --all

 

On Ubuntu or Debian Systems

Use one of the following methods to download the CDH 5 repository or package.

To download and install the CDH 5 "1-click Install" package:

  1. Download the CDH 5 "1-click Install" package:

    OS Version Package Link
    Jessie Jessie package
    Wheezy Wheezy package
    Precise Precise package
    Trusty Trusty package

  2. Install the package by doing one of the following:
    • Choose Open with in the download window to use the package manager.
    • Choose Save File, save the package to a directory to which you have write access (for example, your home directory), and install it from the command line. For example:

      sudo dpkg -i cdh5-repository_1.0_all.deb

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo apt-get update

 

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

OR: To add the CDH 5 repository:

  • Download the appropriate cloudera.list file by issuing one of the following commands. You can use another HTTP client if wget is not available, but the syntax may be different.Important: Ubuntu 14.04 (Trusty)

    For Ubuntu Trusty systems, you must perform an extra step after adding the repository. See "Additional Step for Trusty Ubuntu Trusty and Debian Jessie" below.

     

    OS Version Command
    DebianJessie

    $ sudo wget 'https://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh/cloudera.list' \ -O /etc/apt/sources.list.d/cloudera.list

    DebianWheezy

    $ sudo wget 'https://archive.cloudera.com/cdh5/debian/wheezy/amd64/cdh/cloudera.list' \ -O /etc/apt/sources.list.d/cloudera.list

    UbuntuPrecise

    $ sudo wget 'https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/cloudera.list' \ -O /etc/apt/sources.list.d/cloudera.list

    UbuntuLucid

    $ sudo wget 'https://archive.cloudera.com/cdh5/ubuntu/lucid/amd64/cdh/cloudera.list' \ -O /etc/apt/sources.list.d/cloudera.list

    UbuntuTrusty

    $ sudo wget 'https://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/cloudera.list' \ -O /etc/apt/sources.list.d/cloudera.list

     

Note: Clean repository cache.

Before proceeding, clean cached packages and headers to ensure your system repos are up-to-date:

sudo apt-get update

 

Additional step for Ubuntu Trusty and Debian Jessie

 

This step ensures that you get the right ZooKeeper package for the current CDH release. You need to prioritize the Cloudera repository you have just added, such that you install the CDH version of ZooKeeper rather than the version that is bundled with Ubuntu Trusty or Debian Jessie.

To do this, create a file at /etc/apt/preferences.d/cloudera.pref with the following contents:

Package: *
Pin: release o=Cloudera, l=Cloudera
Pin-Priority: 501

 

Note: You do not need to run apt-get update after creating this file.

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

OR: To build a Debian repository:

If you want to create your own apt repository, create a mirror of the CDH Debian directory and then create an apt repository from the mirror.

Continue with Step 2: Optionally Add a Repository Key. Then choose Step 3: Install CDH 5 with YARN, orStep 4: Install CDH 5 with MRv1; or do both steps to install both implementations.

 

Step 2: Optionally Add a Repository Key

 

Before installing YARN or MRv1: (Optionally) add a repository key on each system in the cluster. Add the Cloudera Public GPG Key to your repository by executing one of the following commands:

  • For RHEL/CentOS/Oracle 5 systems:

    $ sudo rpm --import https://archive.cloudera.com/cdh5/redhat/5/x86_64/cdh/RPM-GPG-KEY-cloudera

  • For RHEL/CentOS/Oracle 6 systems:

    $ sudo rpm --import https://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera

  • For RHEL/CentOS/Oracle 7 systems:

    $ sudo rpm --import https://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/RPM-GPG-KEY-cloudera

  • For all SLES systems:

    $ sudo rpm --import https://archive.cloudera.com/cdh5/sles/11/x86_64/cdh/RPM-GPG-KEY-cloudera

  • For Ubuntu or Debian systems:

    OS Version Command
    Debian Jessie

    $ wget https://archive.cloudera.com/cdh5/debian/jessie/amd64/cdh/archive.key -O archive.key $ sudo apt-key add archive.key

    Debian Wheezy

    $ wget https://archive.cloudera.com/cdh5/debian/wheezy/amd64/cdh/archive.key -O archive.key $ sudo apt-key add archive.key

    Ubuntu Precise

    $ wget https://archive.cloudera.com/cdh5/ubuntu/precise/amd64/cdh/archive.key -O archive.key $ sudo apt-key add archive.key

    Ubuntu Lucid

    $ wget https://archive.cloudera.com/cdh5/ubuntu/lucid/amd64/cdh/archive.key -O archive.key $ sudo apt-key add archive.key

    Ubuntu Trusty

    $ wget https://archive.cloudera.com/cdh5/ubuntu/trusty/amd64/cdh/archive.key -O archive.key $ sudo apt-key add archive.key

This key enables you to verify that you are downloading genuine packages.

 

Step 3: Install CDH 5 with YARN

Note: Skip this step if you intend to use only MRv1. Directions for installing MRv1 are in Step 4.

To install CDH 5 with YARN:

Note: When configuring HA for the NameNode, do not install hadoop-hdfs-secondarynamenode. After completing the HA software configuration, follow the installation instructions under Deploying HDFS High Availability.

  1. Install and deploy ZooKeeper.Important: Cloudera recommends that you install (or update) and start a ZooKeeper cluster before proceeding. This is a requirement if you are deploying high availability (HA) for the NameNode.

    Follow instructions under ZooKeeper Installation.

  2. Install each type of daemon package on the appropriate systems(s), as follows.

    Where to install

    Install commands

    Resource Manager host (analogous to MRv1 JobTracker) running:

     

    RHEL/CentOS compatible

    sudo yum clean all; sudo yum install hadoop-yarn-resourcemanager

    SLES

    sudo zypper clean --all; sudo zypper install hadoop-yarn-resourcemanager

    Ubuntu or Debian

    sudo apt-get update; sudo apt-get install hadoop-yarn-resourcemanager

    NameNode host running:

     

    RHEL/CentOS compatible

    sudo yum clean all; sudo yum install hadoop-hdfs-namenode

    SLES

    sudo zypper clean --all; sudo zypper install hadoop-hdfs-namenode

    Ubuntu or Debian

    sudo apt-get install hadoop-hdfs-namenode

    Secondary NameNode host (if used) running:

     

    RHEL/CentOS compatible

    sudo yum clean all; sudo yum install hadoop-hdfs-secondarynamenode

    SLES

    sudo zypper clean --all; sudo zypper install hadoop-hdfs-secondarynamenode

    Ubuntu or Debian

    sudo apt-get install hadoop-hdfs-secondarynamenode

    All cluster hosts except the Resource Managerrunning:

     

    RHEL/CentOS compatible

    sudo yum clean all; sudo yum install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce

    SLES

    sudo zypper clean --all; sudo zypper install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce

    Ubuntu or Debian

    sudo apt-get install hadoop-yarn-nodemanager hadoop-hdfs-datanode hadoop-mapreduce

    One host in the cluster running:

     

    RHEL/CentOS compatible

    sudo yum clean all; sudo yum install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver

    SLES

    sudo zypper clean --all; sudo zypper install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver

    Ubuntu or Debian

    sudo apt-get install hadoop-mapreduce-historyserver hadoop-yarn-proxyserver

    All client hosts running:

     

    RHEL/CentOS compatible

    sudo yum clean all; sudo yum install hadoop-client

    SLES

    sudo zypper clean --all; sudo zypper install hadoop-client

    Ubuntu or Debian

    sudo apt-get install hadoop-client

Note: The hadoop-yarn and hadoop-hdfs packages are installed on each system automatically as dependencies of the other packages.

 

Step 4: Install CDH 5 with MRv1

Note: If installing both MRv1 and YARN, do not install packages that you already installed in Step 3: Install CDH 5 with YARN. If installing YARN only, skip this step and go to Step 3: Install CDH 5 with YARN.Important: Before proceeding, you need to decide:

  • Whether to configure High Availability (HA) for the NameNode or JobTracker; see the High Availabilityfor more information and instructions.
  • Where to deploy the NameNode, Secondary NameNode, and JobTracker daemons. As a general rule:
    • The NameNode and JobTracker run on the same "master" host unless the cluster is large (more than a few tens of nodes), and the master host (or hosts) should not run the Secondary NameNode (if used), DataNode or TaskTracker services.
    • In a large cluster, it is especially important that the Secondary NameNode (if used) runs on a separate machine from the NameNode.
    • Each node in the cluster except the master host(s) should run the DataNode and TaskTracker services.

If you decide to configure HA for the NameNode, do not install hadoop-hdfs-secondarynamenode. After completing the HA software configuration, follow the installation instructions under Deploying HDFS High Availability.

First, install and deploy ZooKeeper.Important: Cloudera recommends that you install (or update) and start a ZooKeeper cluster before proceeding. This is a requirement if you are deploying high availability (HA) for the NameNode or JobTracker.

Follow instructions under ZooKeeper Installation. Make sure you create the myid file in the data directory, as instructed, if you are starting a ZooKeeper ensemble after a fresh install.

Next, install packages.

Install each type of daemon package on the appropriate systems(s), as follows.Note: Ubuntu systems may try to start the service immediately after you install it. This should fail harmlessly, but you can find information at askubuntu on how to prevent this.

Where to install

Install commands

JobTracker host running:

 

RHEL/CentOS compatible

sudo yum clean all; sudo yum install hadoop-0.20-mapreduce-jobtracker

SLES

sudo zypper clean --all; sudo zypper install hadoop-0.20-mapreduce-jobtracker

Ubuntu or Debian

sudo apt-get update; sudo apt-get install hadoop-0.20-mapreduce-jobtracker

NameNode host running:

 

RHEL/CentOS compatible

sudo yum clean all; sudo yum install hadoop-hdfs-namenode

SLES

sudo zypper clean --all; sudo zypper install hadoop-hdfs-namenode

Ubuntu or Debian

sudo apt-get install hadoop-hdfs-namenode

Secondary NameNode host (if used) running:

 

RHEL/CentOS compatible

sudo yum clean all; sudo yum install hadoop-hdfs-secondarynamenode

SLES

sudo zypper clean --all; sudo zypper install hadoop-hdfs-secondarynamenode

Ubuntu or Debian

sudo apt-get install hadoop-hdfs-secondarynamenode

All cluster hosts except the JobTracker, NameNode, and Secondary (or Standby) NameNode hosts running:

 

RHEL/CentOS compatible

sudo yum clean all; sudo yum install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode

SLES

sudo zypper clean --all; sudo zypper install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode

Ubuntu or Debian

sudo apt-get install hadoop-0.20-mapreduce-tasktracker hadoop-hdfs-datanode

All client hosts running:

 

RHEL/CentOS compatible

sudo yum clean all; sudo yum install hadoop-client

SLES

sudo zypper clean --all; sudo zypper install hadoop-client

Ubuntu or Debian

sudo apt-get install hadoop-client

 

Step 5: (Optional) Install LZO

 

This section explains how to install LZO ( Lempel–Ziv–Oberhumer) compression. For more information, seeChoosing and Configuring Data CompressionNote: If upgrading (rather than installing for the first time), remove the old LZO version first. For example, on a RHEL system:

yum remove hadoop-lzo

 

  1. Add the repository on each host in the cluster. Follow the instructions for your OS version:

    For OS Version Do this
    RHEL/CentOS/Oracle 5 Go to this link and save the file in the /etc/yum.repos.d/ directory.
    RHEL/CentOS/Oracle 6 Go to this link and save the file in the /etc/yum.repos.d/ directory.
    RHEL/CentOS/Oracle 7 Go to this link and save the file in the /etc/yum.repos.d/ directory.
    SLES
    1. Run the following command:

      $ sudo zypper addrepo -f https://archive.cloudera.com/gplextras5/sles/11/x86_64/gplextras/ cloudera-gplextras5.repo

    2. Update your system package index by running:

      $ sudo zypper refresh

    Ubuntu or Debian Go to this link and save the file as/etc/apt/sources.list.d/gplextras.list.Important: Make sure you do not let the file name default tocloudera.list, as that will overwrite your existing cloudera.list.

  2. Install the package on each host as follows:

    For OS version Install commands
    RHEL/CentOS compatible

    sudo yum install hadoop-lzo

    SLES

    sudo zypper install hadoop-lzo

    Ubuntu or Debian

    sudo apt-get install hadoop-lzo

  3. Continue with installing and deploying CDH. As part of the deployment, you will need to do some additional configuration for LZO, as shown under Configuring LZO.Important: Be sure to do this configuration after you have copied the default configuration files to a custom location and set alternatives to point to it.

 

Step 6: Deploy CDH and Install Components

 

Proceed with:

 

Please Read and Accept our Terms

Note: All CDH and Cloudera Manager hosts that make up a logical cluster need to run on the same major OS release to be covered by Cloudera Support.

CDH 5 provides 64-bit packages for RHEL-compatible, SLES, Ubuntu, and Debian systems as listed below.

 

Operating System Version Packages
Red Hat Enterprise Linux (RHEL)-compatible
RHEL (+ SELinux mode in available versions) 5.7 64-bit
  5.10 64-bit
  6.4 64-bit
  6.5 64-bit
  6.6 64-bit
  6.7 64-bit
  7.1 64-bit
  7.2 64-bit
CentOS (+ SELinux mode in available versions) 5.7 64-bit
  5.10 64-bit
  6.4 64-bit
  6.5 64-bit
  6.6 64-bit
  6.7 64-bit
  7.1 64-bit
  7.2 64-bit
Oracle Enterprise Linux (OEL) with Unbreakable Enterprise Kernel (UEK) 5.7 (UEK R2) 64-bit
  5.10 64-bit
  5.11 64-bit
  6.4 (UEK R2) 64-bit
  6.5 (UEK R2, UEK R3) 64-bit
  6.6 (UEK R3) 64-bit
  6.7 (UEK R3) 64-bit
  7.1 64-bit
  7.2 64-bit
SLES
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 2 64-bit
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 3 64-bit
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 4 64-bit
Ubuntu/Debian
Ubuntu Precise 12.04 - Long-Term Support (LTS) 64-bit
  Trusty 14.04 - Long-Term Support (LTS) 64-bit
Debian Wheezy 7.0, 7.1, and 7.8 64-bit

 

Important: Cloudera supports RHEL 7 with the following limitations:

 

Note:

  • Cloudera Enterprise is supported on platforms with Security-Enhanced Linux (SELinux) enabled. Cloudera is not responsible for policy support nor policy enforcement. If you experience issues with SELinux, contact your OS provider.
  • CDH 5.8 DataNode hosts with EMC® DSSD™ D5™ are supported by RHEL 6.6, 7.1, and 7.2.
Selected tab: SupportedOperatingSystems

Component MariaDB MySQL SQLite PostgreSQL Oracle Derby - see Note 5
Oozie 5.5 5.1, 5.5, 5.6, 5.7 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

11gR2, 12c Default
Flume Default (for the JDBC Channel only)
Hue 5.5 5.1, 5.5, 5.6, 5.7

See Note 6

Default 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

11gR2, 12c
Hive/Impala 5.5 5.1, 5.5, 5.6, 5.7

See Note 1

8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

11gR2, 12c Default
Sentry 5.5 5.1, 5.5, 5.6, 5.7

See Note 1

8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

11gR2, 12c
Sqoop 1 5.5 See Note 4 See Note 4 See Note 4
Sqoop 2 5.5 Default

 

Note:

  1. MySQL 5.5 is supported on CDH 5.1. MySQL 5.6 is supported on CDH 5.1 and higher. The InnoDB storage engine must be enabled in the MySQL server.
  2. Cloudera Manager installation fails if GTID-based replication is enabled in MySQL.
  3. PostgreSQL 9.2 is supported on CDH 5.1 and higher. PostgreSQL 9.3 is supported on CDH 5.2 and higher. PostgreSQL 9.4 is supported on CDH 5.5 and higher.
  4. For purposes of transferring data only, Sqoop 1 supports MySQL 5.0 and above, PostgreSQL 8.4 and above, Oracle 10.2 and above, Teradata 13.10 and above, and Netezza TwinFin 5.0 and above. The Sqoop metastore works only with HSQLDB (1.8.0 and higher 1.x versions; the metastore does not work with any HSQLDB 2.x versions).
  5. Derby is supported as shown in the table, but not always recommended. See the pages for individual components in the Cloudera Installation and Upgrade guide for recommendations.
  6. CDH 5 Hue requires the default MySQL version of the operating system on which it is being installed, which is usually MySQL 5.1, 5.5, or 5.6.
Selected tab: SupportedDatabases
CDH 5.8.x is supported with the versions shown in the following table:
Minimum Supported Version Recommended Version Exceptions
1.7.0_55 1.7.0_67, 1.7.0_75, 1.7.0_80 None
1.8.0_31 1.8.0_60 Cloudera recommends that you not use JDK 1.8.0_40.
Selected tab: SupportedJDKVersions

Hue

Hue works with the two most recent versions of the following browsers. Cookies and JavaScript must be on.

  • Chrome
  • Firefox
  • Safari (not supported on Windows)
  • Internet Explorer
Hue could display in older versions and even other browsers, but you might not have access to all of its features.

Selected tab: SupportedBrowsers

CDH requires IPv4. IPv6 is not supported.

See also Configuring Network Names.

 

Multihoming CDH or Cloudera Manager is not supported outside specifically certified Cloudera partner appliances. Cloudera finds that current Hadoop architectures combined with modern network infrastructures and security practices remove the need for multihoming. Multihoming, however, is beneficial internally in appliance form factors to take advantage of high-bandwidth InfiniBand interconnects.

 

Although some subareas of the product may work with unsupported custom multihoming configurations, there are known issues with multihoming. In addition, unknown issues may arise because multihoming is not covered by our test matrix outside the Cloudera-certified partner appliances.

Selected tab: SupportedInternetProtocol

The following components are supported by the indicated versions of Transport Layer Security (TLS):

 

Table 1. Components Supported by TLS

Component

Role Name Port Version
Flume   Avro Source/Sink   TLS 1.2
Flume   Flume HTTP Source/Sink   TLS 1.2
HBase Master HBase Master Web UI Port 60010 TLS 1.2
HDFS NameNode Secure NameNode Web UI Port 50470 TLS 1.2
HDFS Secondary NameNode Secure Secondary NameNode Web UI Port 50495 TLS 1.2
HDFS HttpFS REST Port 14000 TLS 1.1, TLS 1.2
Hive HiveServer2 HiveServer2 Port 10000 TLS 1.2
Hue Hue Server Hue HTTP Port 8888 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon Beeswax Port 21000 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon HiveServer2 Port 21050 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon Backend Port 22000 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon HTTP Server Port 25000 TLS 1.2
Cloudera Impala Impala StateStore StateStore Service Port 24000 TLS 1.2
Cloudera Impala Impala StateStore StateStore HTTP Server Port 25010 TLS 1.2
Cloudera Impala Impala Catalog Server Catalog Server HTTP Server Port 25020 TLS 1.2
Cloudera Impala Impala Catalog Server Catalog Server Service Port 26000 TLS 1.2
Oozie Oozie Server Oozie HTTPS Port 11443 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTP Port 8983 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTPS Port 8985 TLS 1.1, TLS 1.2
YARN ResourceManager ResourceManager Web Application HTTP Port 8090 TLS 1.2
YARN JobHistory Server MRv1 JobHistory Web Application HTTP Port 19890 TLS 1.2
Selected tab: SupportedTransportLayerSecurityVersions
Selected tab: SystemRequirements

What's New in CDH 5.8.0

Operating System Support

  • Operating Systems - Support for Debian 8.2.

 

 

Apache HBase

  • Additional metrics have been added to monitor garbage collection pauses and other external pauses that might cause a server process to momentarily block any request processing.
  • New throughput and Bloom filter metrics have been added to the existing HBase microbenchmarks.
  • Downstream users of CDH who build on top of Apache HBase can now pull in fewer transitive dependencies by relying on shaded client artifacts. To do so, use the maven artifact hbase-shaded-client where you would normally use the hbase-client artifact. All HBase APIs remain the same. Note: In a previous release of CDH, this maven artifact exists but does not contain any of the needed classes to interact with an HBase cluster.
  • The HMaster Web UI now shows the aggregate of all space being used by snapshots.

Hue

General Features:
  • Rebase of Hue on upstream Hue 3.10.
  • Refactor of Hue Infrastructure:
    • Performance is further optimized for large numbers of databases and tables.
    • Exporting and importing documents is improved.
SQL Editor and Browser:
  • Revamp of the Hue SQL Application:
    • Editor is redesigned to create a single-page experience.
    • Code editor is redesigned with enhanced auto-complete, keyboard shorts, search and replace, and more.
    • A live status of the query history displays as an icon.
Admin and Security:
  • HUE-3386: Users are auto-logged out when TTL Expires. See Securing Sessions.
  • Hue-3808: Users can do live DEBUG log toggling. See Enable DEBUG.
  • A new step in the Cloudera Manager Add Service wizard helps configure and test an external Hue database.
Search:
Oozie:
  • Hue-3464: The dashboard and editor are decoupled for granular access.
  • Saved Hive queries can be dragged and dropped into a workflow.

Apache Oozie

  • OOZIE-2330 : The Spark Action now allows <file> and <archive> elements. It also omits <job-tracker> and <name-node> elements to allow for the use of the global or default values.
  • The launcher job no longer uses YARN's uber mode by default.

Cloudera Search

Apache Sentry

  • Sentry adds support for securing data on Amazon S3. As a result, Sentry will now be able to secure URIs with an S3 schema.
  • Cloudera Search adds support for storing permissions in the Sentry service. You can enable storing permissions in the Sentry service by Enabling Sentry Policy File Authorization for Solr. If you have already configured Sentry's policy file-based approach, you can migrate existing authorization settings as described in Migrating from Sentry Policy Files to the Sentry Service. solrctl has been extended to support:
    • Migrating existing policy files to the Sentry service
    • Managing managing permissions in the Sentry service
  • SENTRY-1175: Improved usability for Sentry URIs and URI privileges. If URIs in Hive DDL statements or URI privileges lack scheme and authority components, Sentry automatically completes such URIs by applying the default scheme and authority based on the HDFS configuration provided to Sentry.
  • Performance Improvements
    • SENTRY-1293: ResourceAuthorizationProvider.doHasAccess no longer performs expensive operations to convert string permissions to Privilege objects.
    • SENTRY-1292: Reordered the DBModelAction EnumSet to improve authorization performance.

 

 

 

Selected tab: WhatsNew

Want to Get Involved or Learn More?

Check out our other resources

Cloudera Community

Collaborate with your peers, industry experts, and Clouderans to make the most of your investment in Hadoop.

Cloudera University

Receive expert Hadoop training through Cloudera University, the industry's only truly dynamic Hadoop training curriculum that’s updated regularly to reflect the state of the art in big data.