Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×

Please Read and Accept our Terms


Long term component architecture

As the main curator of open standards in Hadoop, Cloudera has a track record of bringing new open source solutions into its platform (such as Apache Spark, Apache HBase, and Apache Parquet) that are eventually adopted by the community at large. As standards, you can build longterm architecture on these components with confidence.

 

PLEASE NOTE:

With the exception of DSSD support, Cloudera Enterprise 5.6.0 is identical to CDH 5.5.2/Cloudera Manager 5.5.3  If you do not need DSSD support, you do not need to upgrade if you are already using the latest 5.5.x release.

 

Note: Mixed operating system type and version clusters are supported, however using the same version of the same operating system on all cluster hosts is strongly recommended.

CDH 5 provides 64-bit packages for RHEL-compatible, SLES, Ubuntu, and Debian systems as listed below.

 

Operating System Version Packages
Red Hat Enterprise Linux (RHEL)-compatible
RHEL (+ SELinux mode in available versions) 5.7 64-bit
  5.10 64-bit
  6.4 64-bit
  6.5 64-bit
  6.6 64-bit
  6.7 64-bit
  7.1 64-bit
  7.2 64-bit
CentOS (+ SELinux mode in available versions) 5.7 64-bit
  5.10 64-bit
  6.4 64-bit
  6.5 64-bit
  6.6 64-bit
  6.7 64-bit
  7.1 64-bit
  7.2 64-bit
Oracle Linux with default kernel and Unbreakable Enterprise Kernel 5.7 (UEK R2) 64-bit
  5.10 64-bit
  5.11 64-bit
  6.4 (UEK R2) 64-bit
  6.5 (UEK R2, UEK R3) 64-bit
  6.6 (UEK R3) 64-bit
  6.7 (UEK R3) 64-bit
  7.1 64-bit
  7.2 64-bit
SLES
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 2 64-bit
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 3 64-bit
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 4 64-bit
Ubuntu/Debian
Ubuntu Precise 12.04 - Long-Term Support (LTS) 64-bit
  Trusty 14.04 - Long-Term Support (LTS) 64-bit
Debian Wheezy 7.0, 7.1, and 7.8 64-bit
 
Important: Cloudera supports RHEL 7 with the following limitations:
 
Note:
  • Cloudera Enterprise is supported on platforms with Security-Enhanced Linux (SELinux) enabled. Cloudera is not responsible for policy support nor policy enforcement. If you experience issues with SELinux, contact your OS provider.
  • CDH 5.7 DataNode hosts with EMC® DSSD™ D5™ are supported by RHEL 6.6, 7.1, and 7.2. CDH 5.6 DataNode hosts with EMC® DSSD™ D5™ are only supported by RHEL 6.6.
Selected tab: SupportedOperatingSystems
Component MariaDB MySQL SQLite PostgreSQL Oracle Derby - see Note 5
Oozie 5.5 5.1, 5.5, 5.6, 5.7 – 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

 11gR2, 12c Default
Flume – – – – – Default (for the JDBC Channel only)
Hue 5.5 5.1, 5.5, 5.6, 5.7

See Note 6

 Default 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

 11gR2, 12c –
Hive/Impala 5.5 5.1, 5.5, 5.6, 5.7

See Note 1

 – 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

 11gR2, 12c Default
Sentry 5.5 5.1, 5.5, 5.6, 5.7

See Note 1

 – 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4

See Note 3

 11gR2, 12c –
Sqoop 1 5.5 See Note 4 – See Note 4 See Note 4 –
Sqoop 2 5.5 – – – – Default
 
  Note:
  1. MySQL 5.5 is supported on CDH 5.1. MySQL 5.6 is supported on CDH 5.1 and higher. The InnoDB storage engine must be enabled in the MySQL server.
  2. Cloudera Manager installation fails if GTID-based replication is enabled in MySQL.
  3. PostgreSQL 9.2 is supported on CDH 5.1 and higher. PostgreSQL 9.3 is supported on CDH 5.2 and higher. PostgreSQL 9.4 is supported on CDH 5.5 and higher.
  4. For purposes of transferring data only, Sqoop 1 supports MySQL 5.0 and above, PostgreSQL 8.4 and above, Oracle 10.2 and above, Teradata 13.10 and above, and Netezza TwinFin 5.0 and above. The Sqoop metastore works only with HSQLDB (1.8.0 and higher 1.x versions; the metastore does not work with any HSQLDB 2.x versions).
  5. Derby is supported as shown in the table, but not always recommended. See the pages for individual components in the Cloudera Installation and Upgrade guide for recommendations.
  6. CDH 5 Hue requires the default MySQL version of the operating system on which it is being installed, which is usually MySQL 5.1, 5.5, or 5.6.
Selected tab: SupportedDatabases
  Important: JDK 1.6 is not supported on any CDH 5 release (even though the libraries of CDH 5.0-CDH 5.4 are compatible). Applications using CDH libraries must run a supported version of JDK 1.7 or higher, and one that also matches the JDK version of your CDH cluster.
 
CDH 5.7.x is supported with the versions shown in the following table:
Minimum Supported Version Recommended Version Exceptions
1.7.0_55 1.7.0_67, 1.7.0_75, 1.7.0_80 None
1.8.0_31 1.8.0_60 Cloudera recommends that you not use JDK 1.8.0_40.
Selected tab: SupportedJDKVersions

Hue

Hue works with the two most recent versions of the following browsers. Cookies and JavaScript must be on.

  • Chrome
  • Firefox
  • Safari (not supported on Windows)
  • Internet Explorer

Hue could display in older versions and even other browsers, but you might not have access to all of its features.

Selected tab: SupportedBrowsers

 

CDH requires IPv4. IPv6 is not supported.

See also Configuring Network Names.

 

Selected tab: SupportedInternetProtocol

The following components are supported by the indicated versions of Transport Layer Security (TLS):

 

Table 1. Components Supported by TLS

Component

 Role Name Port Version
Flume   Avro Source/Sink   TLS 1.2
Flume   Flume HTTP Source/Sink   TLS 1.2
HBase Master HBase Master Web UI Port 60010 TLS 1.2
HDFS NameNode Secure NameNode Web UI Port 50470 TLS 1.2
HDFS Secondary NameNode Secure Secondary NameNode Web UI Port 50495 TLS 1.2
HDFS HttpFS REST Port 14000 TLS 1.1, TLS 1.2
Hive HiveServer2 HiveServer2 Port 10000 TLS 1.2
Hue Hue Server Hue HTTP Port 8888 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon Beeswax Port 21000 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon HiveServer2 Port 21050 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon Backend Port 22000 TLS 1.2
Cloudera Impala Impala Daemon Impala Daemon HTTP Server Port 25000 TLS 1.2
Cloudera Impala Impala StateStore StateStore Service Port 24000 TLS 1.2
Cloudera Impala Impala StateStore StateStore HTTP Server Port 25010 TLS 1.2
Cloudera Impala Impala Catalog Server Catalog Server HTTP Server Port 25020 TLS 1.2
Cloudera Impala Impala Catalog Server Catalog Server Service Port 26000 TLS 1.2
Oozie Oozie Server Oozie HTTPS Port 11443 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTP Port 8983 TLS 1.1, TLS 1.2
Solr Solr Server Solr HTTPS Port 8985 TLS 1.1, TLS 1.2
YARN ResourceManager ResourceManager Web Application HTTP Port 8090 TLS 1.2
YARN JobHistory Server MRv1 JobHistory Web Application HTTP Port 19890 TLS 1.2
Selected tab: SupportedTransportLayerSecurityVersions
Selected tab: SystemRequirements

Issues Fixed in CDH 5.7.5

Upstream Issues Fixed

The following upstream issues are fixed in CDH 5.7.5:

  • HADOOP-10300 - Allowed deferred sending of call responses
  • HADOOP-12483 - Maintain wrapped SASL ordering for postponed IPC responses
  • HADOOP-13317 - Add logs to KMS server-side to improve supportability
  • HADOOP-13558 - UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket
  • HADOOP-13638 - KMS should set UGI's Configuration object properly
  • HADOOP-13669 - KMS Server should log exceptions before throwing
  • HADOOP-13693 - Remove the message about HTTP OPTIONS in SPNEGO initialization message from kms audit log
  • HDFS-4176 - EditLogTailer should call rollEdits with a timeout
  • HDFS-6962 - ACLs inheritance conflict with umaskmode
  • HDFS-7413 - Some unit tests should use NameNodeProtocols instead of FSNameSystem
  • HDFS-7964 - Add support for async edit logging
  • HDFS-8224 - Schedule a block for scanning if its metadata file is corrupt
  • HDFS-8709 - Clarify automatic sync in FSEditLog#logEdit
  • HDFS-9038 - DFS reserved space is erroneously counted towards non-DFS used
  • HDFS-10178 - Permanent write failures can happen if pipeline recoveries occur for the first packet
  • HDFS-10609 - Uncaught InvalidEncryptionKeyException during pipeline recovery can abort downstream applications
  • HDFS-10641 - TestBlockManager#testBlockReportQueueing fails intermittently
  • HDFS-10722 - Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs
  • HDFS-10760 - DataXceiver#run() should not log InvalidToken exception as an error
  • HDFS-10879 - TestEncryptionZonesWithKMS#testReadWrite fails intermittently
  • HDFS-10962 - TestRequestHedgingProxyProvider fails intermittently
  • HDFS-11012 - Unnecessary INFO logging on DFSClients for InvalidToken
  • MAPREDUCE-6633 - AM should retry map attempts if the reduce task encounters commpression related errors
  • MAPREDUCE-6718 - Add progress log to JHS during startup
  • MAPREDUCE-6728 - Give fetchers hint when ShuffleHandler rejects a shuffling connection
  • MAPREDUCE-6771 - RMContainerAllocator sends container diagnostics event after corresponding completion event
  • YARN-4004 - container-executor should print output of Docker logs if the Docker container exits with non-0 exit status
  • YARN-4017 - container-executor overuses PATH_MAX
  • YARN-4245 - Generalize config file handling in container-executor
  • YARN-4255 - container-executor does not clean up Docker operation command files
  • YARN-4723 - NodesListManager$UnknownNodeId ClassCastException
  • YARN-4940 - YARN node -list -all fails if RM starts with decommissioned node
  • YARN-5704 - Provide configuration knobs to control enabling/disabling new/work in progress features in container-executor
  • HBASE-16294 - hbck reporting "No HDFS region dir found" for replicas
  • HBASE-16699 - Overflows in AverageIntervalRateLimiter's refill() and getWaitInterval()
  • HBASE-16767 - Mob compaction needs to clean up files in /hbase/mobdir/.tmp and /hbase/mobdir/.tmp/.bulkload when running into IO exceptions
  • HIVE-10384 - BackportRetryingMetaStoreClient does not retry wrapped TTransportExceptions
  • HIVE-12077 - MSCK Repair table should fix partitions in batches
  • HIVE-12475 - Parquet schema evolution within array<struct<>> does not work
  • HIVE-12785 - View with union type and UDF to the struct is broken
  • HIVE-13058 - Add session and operation_log directory deletion messages
  • HIVE-13198 - Authorization issues with cascading views
  • HIVE-13237 - Select parquet struct field with upper case throws NPE
  • HIVE-13429 - Tool to remove dangling scratch dir
  • HIVE-13997 - Insert overwrite directory does not overwrite existing files
  • HIVE-14313 - Test failure TestMetaStoreMetrics.testConnections
  • HIVE-14421 - FS.deleteOnExit holds references to _tmp_space.db files
  • HIVE-14762 - Add logging while removing scratch space
  • HIVE-14784 - Operation logs are disabled automatically if the parent directory does not exist
  • HIVE-14799 - Query operations are not thread safe during cancellation
  • HIVE-14805 - Subquery inside a view will have the object in the subquery as the direct input
  • HIVE-14810 - Fix failing test: TestMetaStoreMetrics.testMetaDataCounts
  • HIVE-14817 - Shutdown the SessionManager timeoutChecker thread properly upon shutdown
  • HIVE-14839 - Improve the stability of TestSessionManagerMetrics
  • HUE-3860 - Fix unittest beeswax.tests.test_hiveserver2_jdbc_url
  • HUE-3905 - Reset beeswax.conf params in beeswax.tests:test_hiveserver2_jdbc_url
  • HUE-4201 - Add warning about max limit of cells before truncation in the download query result
  • HUE-4662 - Fixed: Wildcard Certificates not supported
  • HUE-4739 - Fixed Jobbrowser tests which were failing after resource manager pool change
  • HUE-4916 - Truncate last name to 30 chars on ldap import
  • HUE-4968 - Remove access to /oozie/import_wokflow when v2 is enabled
  • HUE-5042 - Unable to kill jobs after Resource Manager failover
  • HUE-5050 - Logout fails for local login when multiple backends are used
  • HUE-5161 - Speed up roles rendering
  • HUE-5163 - Speed up initial page rendering
  • IMPALA-1619 - Support 64-bit allocations
  • IMPALA-1740 - Add support for skip.header.line.count
  • IMPALA-3458 - Fix table creation to test insert with header lines
  • IMPALA-3949 - Log the error message in FileSystemUtil.copyToLocal()
  • IMPALA-4037 - Fx locking during query cancellation
  • IMPALA-4076 - Fix runtime filter sort compare method
  • IMPALA-4099 - Fix the error message while loading UDFs with no JARs
  • IMPALA-4120 - Incorrect results with LEAD() analytic function
  • IMPALA-4135 - Thrift threaded server times-out connections during high load
  • IMPALA-4170 - Fix identifier quoting in COMPUTE INCREMENTAL STATS
  • IMPALA-4196 - Cross compile bit-byte functions
  • IMPALA-4237 - Fix materialization of 4 byte decimals in data source scan node
  • IMPALA-4246 - SleepForMs() utility function has undefined behavior for > 1s
  • OOZIE-1814 - Oozie should mask any passwords in logs and REST interfaces
  • SOLR-9310 - PeerSync fails on a node restart due to IndexFingerPrint mismatch
  • SPARK-12009 - Avoid re-allocating YARN container when driver wants to stop all Executors
  • SPARK-12392 - Optimize a location order of broadcast blocks by considering preferred local hosts
  • SPARK-12941 - Spark-SQL JDBC Oracle dialect fails to map string datatypes to Oracle VARCHAR datatype mapping
  • SPARK-13328 - Poor read performance for broadcast variables with dynamic resource allocation
  • SPARK-16625 - General data types to be mapped to Oracle
  • SPARK-16711 - YarnShuffleService does not re-init properly on YARN rolling upgrade
  • SPARK-17171 - DAG lists all partitions in the graph
  • SPARK-17433 - YarnShuffleService does not handle moving credentials levelDb
  • SPARK-17611 - Make shuffle service test really test authentication
  • SPARK-17644 - Do not add failedStages when abortStage for fetch failure
  • SPARK-17696 - Partial backport of to branch-1.6.
  • SQOOP-2952 - Row key not added into column family using --hbase-bulkload
  • SQOOP-2986 - Add validation check for --hive-import and --incremental lastmodified
  • SQOOP-3021 - ClassWriter fails if a column name contains a backslash character

 

Selected tab: WhatsNew
Selected tab: Documentation

Want to Get Involved or Learn More?

Check out our other resources

Cloudera Community

Collaborate with your peers, industry experts, and Clouderans to make the most of your investment in Hadoop.

Cloudera University

Receive expert Hadoop training through Cloudera University, the industry's only truly dynamic Hadoop training curriculum that’s updated regularly to reflect the state of the art in big data.