Long term component architecture
As the main curator of open standards in Hadoop, Cloudera has a track record of bringing new open source solutions into its platform (such as Apache Spark, Apache HBase, and Apache Parquet) that are eventually adopted by the community at large. As standards, you can build longterm architecture on these components with confidence.
With the exception of DSSD support, Cloudera Enterprise 5.6.0 is identical to CDH 5.5.2/Cloudera Manager 5.5.3 If you do not need DSSD support, you do not need to upgrade if you are already using the latest 5.5.x release.
- System Requirements
- What's New
- Supported Operating Systems
- Supported Databases
- Supported JDK Versions
- Supported Internet Protocol
Supported Operating Systems
CDH 5 provides packages for Red-Hat-compatible, SLES, Ubuntu, and Debian systems as described below.
|Red Hat Enterprise Linux (RHEL)-compatible|
|Red Hat Enterprise Linux||5.7||64-bit|
|6.4 in SE Linux mode||64-bit|
|6.4 in SE Linux mode||64-bit|
|Oracle Linux with default kernel and Unbreakable Enterprise Kernel||5.6 (UEK R2)||64-bit|
|6.4 (UEK R2)||64-bit|
|6.5 (UEK R2, UEK R3)||64-bit|
|SLES Linux Enterprise Server (SLES)||11 with Service Pack 2 or later||64-bit|
|Ubuntu||Precise (12.04) - Long-Term Support (LTS)||64-bit|
|Trusty (14.04) - Long-Term Support (LTS)||64-bit|
|Debian||Wheezy (7.0, 7.1)||64-bit|
- CDH 5 provides only 64-bit packages.
- Cloudera has received reports that our RPMs work well on Fedora, but we have not tested this.
- If you are using an operating system that is not supported by Cloudera packages, you can also download source tarballs from Downloads.
|Component||MySQL||SQLite||PostgreSQL||Oracle||Derby - see Note 4|
|Oozie||5.5, 5.6||-||8.4, 9.1, 9.2, 9.3
See Note 2
|Flume||-||-||-||-||Default (for the JDBC Channel only)|
See Note 1
|Default||8.4, 9.1, 9.2, 9.3
See Note 2
See Note 1
|-||8.4, 9.1, 9.2, 9.3
See Note 2
See Note 1
|-||8.4, 9.1, 9.2,, 9.3
See Note 2
|Sqoop 1||See Note 3||-||See Note 3||See Note 3||-|
|Sqoop 2||See Note 4||-||See Note 4||See Note 4||Default|
- MySQL 5.5 is supported on CDH 5.1. MySQL 5.6 is supported on CDH 5.1 and later.
- PostgreSQL 9.2 is supported on CDH 5.1 and later. PostgreSQL 9.3 is supported on CDH 5.2 and later.
- For the purposes of transferring data only, Sqoop 1 supports MySQL 5.0 and above, PostgreSQL 8.4 and above, Oracle 10.2 and above, Teradata 13.10 and above, and Netezza TwinFin 5.0 and above. The Sqoop metastore works only with HSQLDB (1.8.0 and higher 1.x versions; the metastore does not work with any HSQLDB 2.x versions).
- Sqoop 2 can transfer data to and from MySQL 5.0 and above, PostgreSQL 8.4 and above, Oracle 10.2 and above, and Microsoft SQL Server 2012 and above. The Sqoop 2 repository database is supported only on Derby.
- Derby is supported as shown in the table, but not always recommended. See the pages for individual components in the Cloudera Installation and Upgrade guide for recommendations.
Supported JDK Versions
CDH 5 is supported with the versions shown in the table that follows.
Table 1. Supported JDK Versions
|Latest Certified Version||Minimum Supported Version||Exceptions|
Supported Internet Protocol
Known Issues Fixed in CDH 5.3.9
Apache Commons Collections deserialization vulnerability
Cloudera has learned of a potential security vulnerability in a third-party library called the Apache Commons Collections. This library is used in products distributed and supported by Cloudera (“Cloudera Products”), including core Apache Hadoop. The Apache Commons Collections library is also in widespread use beyond the Hadoop ecosystem. At this time, no specific attack vector for this vulnerability has been identified as present in Cloudera Products.
In an abundance of caution, we are currently in the process of incorporating a version of the Apache Commons Collections library with a fix into the Cloudera Products. In most cases, this will require coordination with the projects in the Apache community. One example of this is tracked by HADOOP-12577.
The Apache Commons Collections potential security vulnerability is titled “Arbitrary remote code execution with InvokerTransformer” and is tracked by COLLECTIONS-580. MITRE has not issued a CVE, but related CVE-2015-4852 has been filed for the vulnerability. CERT has issued Vulnerability Note #576313 for this issue.
Releases affected: CDH 5.5.0, CDH 5.4.8 and lower, CDH 5.3.8 and lower, Cloudera Manager 5.5.0, Cloudera Manager 5.4.8 and lower, Cloudera Manager 5.3.8 and lower, Cloudera Navigator 2.4.0, Cloudera Navigator 2.3.8 and lower
Users affected: All
Severity (Low/Medium/High): High
Impact: This potential vulnerability may enable an attacker to execute arbitrary code from a remote machine without requiring authentication.
Immediate action required: Upgrade to Cloudera Manager 5.3.9 and CDH 5.3.9.
Upstream Issues Fixed
The following upstream issues are fixed in CDH 5.3.9:
- FLUME-2841 - Upgrade commons-collections to 3.2.2
- HADOOP-12577 - Bumped up commons-collections version to 3.2.2 to address a security flaw
- HDFS-7785 - Improve diagnostics information for HttpPutFailedException
- HDFS-7798 - Checkpointing failure caused by shared KerberosAuthenticator
- HDFS-7871 - NameNodeEditLogRoller can keep printing 'Swallowing exception' message
- HDFS-9123 - Copying from the root to a subdirectory should be forbidden
- HDFS-9273 - ACLs on root directory may be lost after NN restart
- HDFS-9332 - Fix Precondition failures from NameNodeEditLogRoller while saving namespace
- HDFS-9470 - Encryption zone on root not loaded from fsimage after NN restart
- MAPREDUCE-6191 - Improve clearing stale state of Java serialization testcase
- MAPREDUCE-6233 - org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk
- MAPREDUCE-6549 - Multibyte delimiters with LineRecordReader cause duplicate records
- YARN-3564 - Fix TestContainerAllocation.testAMContainerAllocationWhenDNSUnavailable fails randomly
- YARN-3602 - TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IOException from cleanup
- YARN-3675 - FairScheduler: RM quits when node removal races with continuous-scheduling on the same node
- HBASE-13134 - mutateRow and checkAndMutate APIs do not throw region level exceptions
- HBASE-14196 - Thrift server idle connection timeout issue
- HBASE-14283 - Reverse scan does not work with HFile inline index/bloom blocks
- HBASE-14533 - Thrift client gets "AsyncProcess: Failed to get region location .... closed"
- HBASE-14799 - Commons-collections object deserialization remote command execution vulnerability
- HIVE-6099 - Multi insert does not work properly with distinct count
- HIVE-7146 - posexplode() UDTF fails with a NullPointerException on NULL columns
- HIVE-8612 - Support metadata result filter hooks
- HIVE-9475 - HiveMetastoreClient.tableExists does not work
- HIVE-10895 - ObjectStore does not close Query objects in some calls, causing a potential leak in some metastore db resources
- HIVE-11255 - get_table_objects_by_name() in HiveMetaStore.java needs to retrieve table objects in multiple batches
- HIVE-12378 - Exception on HBaseSerDe.serialize binary field
- HUE-3035 - [beeswax] Optimize sample data query for partitioned tables
- IMPALA-1746 - QueryExecState does not check for query cancellation or errors
- IMPALA-1756 - Constant filter expressions are not checked for errors and state cleanup not done before throwing exception
- IMPALA-1917 - DCHECK on destroying an ExprContext
- IMPALA-2141 - UnionNode::GetNext() does not check for query errors
- IMPALA-2264 - Fix edge cases for decimal/integer cast
- IMPALA-2514 - DCHECK on destroying an ExprContext
- OOZIE-2413 - Kerberos credentials can expire if the KDC is slow to respond
- PIG-3641 - Split "otherwise" producing incorrect output when combined with ColumnPruning
- SPARK-11484 - [WEBUI] Using proxyBase set by spark instead of env
- SPARK-11652 - [CORE] Remote code execution with InvokerTransformer
Want to Get Involved or Learn More?
Check out our other resources
Receive expert Hadoop training through Cloudera University, the industry's only truly dynamic Hadoop training curriculum that’s updated regularly to reflect the state of the art in big data.