Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×

Please Read and Accept our Terms


Long term component architecture

As the main curator of open standards in Hadoop, Cloudera has a track record of bringing new open source solutions into its platform (such as Apache Spark, Apache HBase, and Apache Parquet) that are eventually adopted by the community at large. As standards, you can build longterm architecture on these components with confidence.

 

PLEASE NOTE:

With the exception of DSSD support, Cloudera Enterprise 5.6.0 is identical to CDH 5.5.2/Cloudera Manager 5.5.3  If you do not need DSSD support, you do not need to upgrade if you are already using the latest 5.5.x release.

 

CDH 5 provides packages for Red-Hat-compatible, SLES, Ubuntu, and Debian systems as described below.

Operating System Version Packages
Red Hat Enterprise Linux (RHEL)-compatible
Red Hat Enterprise Linux 5.7 64-bit
  5.10 64-bit
  6.4 64-bit
  6.5 64-bit
  6.5 in SE Linux mode 64-bit
  6.6 64-bit
CentOS 5.7 64-bit
  5.10 64-bit
  6.4 64-bit
  6.5 64-bit
  6.5 in SE Linux mode 64-bit
  6.6 64-bit
Oracle Linux with default kernel and Unbreakable Enterprise Kernel 5.6 (UEK R2) 64-bit
  6.4 (UEK R2) 64-bit
  6.5 (UEK R2, UEK R3) 64-bit
  6.6 (UEK R3) 64-bit
SLES
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 2 64-bit
SUSE Linux Enterprise Server (SLES) 11 with Service Pack 3 64-bit
Ubuntu/Debian
Ubuntu Precise (12.04) - Long-Term Support (LTS) 64-bit
  Trusty (14.04) - Long-Term Support (LTS) 64-bit
Debian Wheezy (7.0) 64-bit

Note:

  • CDH 5 provides only 64-bit packages.
  • Cloudera has received reports that our RPMs work well on Fedora, but we have not tested this.
  • If you are using an operating system that is not supported by Cloudera packages, you can also download source tarballs from Downloads.
Selected tab: SupportedOperatingSystems
Component MySQL SQLite PostgreSQL Oracle Derby - see Note 4
Oozie 5.5, 5.6 - 8.4, 9.2, 9.3

See Note 2

11gR2 Default
Flume - - - - Default (for the JDBC Channel only)
Hue 5.5, 5.6

See Note 1

Default 8.4, 9.2, 9.3

See Note 2

11gR2 -
Hive/Impala 5.5, 5.6

See Note 1

- 8.4, 9.2, 9.3

See Note 2

11gR2 Default
Sentry 5.5, 5.6

See Note 1

- 8.4, 9.2, 9.3

See Note 2

11gR2 -
Sqoop 1 See Note 3 - See Note 3 See Note 3 -
Sqoop 2 See Note 4 - See Note 4 See Note 4 Default

Note:

  1. MySQL 5.5 is supported on CDH 5.1. MySQL 5.6 is supported on CDH 5.1 and later. The InnoDB storage engine must be enabled in the MySQL server.
  2. PostgreSQL 9.2 is supported on CDH 5.1 and later. PostgreSQL 9.3 is supported on CDH 5.2 and later.
  3. For the purposes of transferring data only, Sqoop 1 supports MySQL 5.0 and above, PostgreSQL 8.4 and above, Oracle 10.2 and above, Teradata 13.10 and above, and Netezza TwinFin 5.0 and above. The Sqoop metastore works only with HSQLDB (1.8.0 and higher 1.x versions; the metastore does not work with any HSQLDB 2.x versions).
  4. Sqoop 2 can transfer data to and from MySQL 5.0 and above, PostgreSQL 8.4 and above, Oracle 10.2 and above, and Microsoft SQL Server 2012 and above. The Sqoop 2 repository database is supported only on Derby and PostgreSQL.
  5. Derby is supported as shown in the table, but not always recommended. See the pages for individual components in the Cloudera Installation and Upgrade guide for recommendations.
Selected tab: SupportedDatabases
CDH 5.4.x is supported with the versions shown in the following table:
Minimum Supported Version Recommended Version Notes
1.7.0_55 1.7.0_67 or 1.7.0_75 None
1.8.0_60 1.8.0_60 None
Selected tab: SupportedJDKVersions

CDH requires IPv4. IPv6 is not supported.

See also Configuring Network Names.

Selected tab: SupportedInternetProtocol
Selected tab: SystemRequirements

 

Issues Fixed in CDH 5.4.10

Upstream Issues Fixed

The following upstream issues are fixed in CDH 5.4.10:

  • FLUME-2712 - Optional channel errors slows down the Source to Main channel event rate
  • FLUME-2886 - Optional Channels can cause OOMs
  • HADOOP-10406 - TestIPC.testIpcWithReaderQueuing may fail
  • HADOOP-10668 - TestZKFailoverControllerStress#testExpireBackAndForth occasionally fails
  • HADOOP-11218 - Add TLSv1.1,TLSv1.2 to KMS, HttpFS, SSLFactory
  • HADOOP-12200 - TestCryptoStreamsWithOpensslAesCtrCryptoCodec should be skipped in non-native profile
  • HADOOP-12240 - Fix tests requiring native library to be skipped in non-native profile
  • HADOOP-12280 - Skip unit tests based on Maven profile rather than NativeCodeLoader.isNativeCodeLoaded
  • HADOOP-12417 - TestWebDelegationToken failing with port in use
  • HADOOP-12418 - TestRPC.testRPCInterruptedSimple fails intermittently
  • HADOOP-12464 - Interrupted client may try to fail-over and retry
  • HADOOP-12468 - Partial group resolution failure should not result in user lockout.
  • HADOOP-12474 - MiniKMS should use random ports for Jetty server by default
  • HADOOP-12559 - KMS connection failures should trigger TGT renewal
  • HADOOP-12604 - Exception may be swallowed in KMSClientProvider.
  • HADOOP-12605 - Fix intermittent failure of TestIPC.testIpcWithReaderQueuing
  • HADOOP-12668 - Support excluding weak Ciphers in HttpServer2 through ssl-server.conf
  • HADOOP-12682 - Fix TestKMS#testKMSRestart* failure
  • HADOOP-12699 - TestKMS#testKMSProvider intermittently fails during 'test rollover draining'
  • HADOOP-12715 - TestValueQueue#testgetAtMostPolicyALL fails intermittently
  • HADOOP-12736 - TestTimedOutTestsListener#testThreadDumpAndDeadlocks sometimes times out
  • HADOOP-12788 - OpensslAesCtrCryptoCodec should log which random number generator is used
  • HDFS-6533 - TestBPOfferService#testBasicFunctionalitytest fails intermittently
  • HDFS-7553 - fix the TestDFSUpgradeWithHA due to BindException
  • HDFS-8647 - Abstract BlockManager's rack policy into BlockPlacementPolicy
  • HDFS-9083 - Replication violates block placement policy
  • HDFS-9092 - Nfs silently drops overlapping write requests and causes data copying to fail
  • HDFS-9289 - Make DataStreamer#block thread safe and verify genStamp in commitBlock
  • HDFS-9313 - Possible NullPointerException in BlockManager if no excess replica can be chosen
  • HDFS-9347 - Invariant assumption in TestQuorumJournalManager.shutdown() is wrong
  • HDFS-9358 - TestNodeCount#testNodeCount timed out
  • HDFS-9406 - FSImage may get corrupted after deleting snapshot
  • HDFS-9445 - Datanode may deadlock while handling a bad volume
  • HDFS-9688 - Test the effect of nested encryption zones in HDFS downgrade
  • HDFS-9721 - Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference
  • MAPREDUCE-6302 - Incorrect headroom can lead to a deadlock between map and reduce allocations
  • MAPREDUCE-6460 - TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails
  • YARN-2902 - Killing a container that is localizing can orphan resources in the DOWNLOADING state
  • YARN-4155 - TestLogAggregationService.testLogAggregationServiceWithInterval failing
  • YARN-4204 - ConcurrentModificationException in FairSchedulerQueueInfo
  • YARN-4347 - Resource manager fails with Null pointer exception
  • YARN-4380 - TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently
  • YARN-4393 - Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease
  • YARN-4573 - Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled
  • YARN-4613 - Fix test failure in TestClientRMService#testGetClusterNodes
  • HBASE-14205 - RegionCoprocessorHost System.nanoTime() performance bottleneck
  • HBASE-14621 - ReplicationLogCleaner stuck on RS crash
  • HBASE-14923 - VerifyReplication should not mask the exception during result comparison
  • HBASE-14926 - Hung ThriftServer; no timeout on read from client; if client crashes, worker thread gets stuck reading
  • HBASE-15019 - Replication stuck when HDFS is restarted
  • HBASE-15031 - Fix merge of MVCC and SequenceID performance regression in branch-1.0
  • HBASE-15032 - hbase shell scan filter string assumes UTF-8 encoding
  • HBASE-15035 - bulkloading hfiles with tags that require splits do not preserve tags
  • HBASE-15052 - Use EnvironmentEdgeManager in ReplicationSource
  • HBASE-15104 - Occasional failures due to NotServingRegionException in IT tests
  • HBASE-15157 - Add *PerformanceTest for Append, CheckAnd*
  • HBASE-15213 - Fix increment performance regression caused by HBASE-8763 on branch-1.0
  • HIVE-7575 - GetTables thrift call is very slow
  • HIVE-10213 - MapReduce jobs using dynamic-partitioning fail on commit
  • HIVE-10514 - Fix MiniCliDriver tests failure
  • HIVE-11826 - 'hadoop.proxyuser.hive.groups' configuration doesn't prevent unauthorized user to access metastore
  • HIVE-11828 - beeline -f fails on scripts with tabs between column type and comment
  • HIVE-11977 - Hive should handle an external avro table with zero length files present
  • HIVE-12008 - Hive queries failing when using count(*) on column in view
  • HIVE-12388 - GetTables cannot get external tables when TABLE type argument is given
  • HIVE-12505 - Insert overwrite in same encrypted zone silently fails to remove some existing files
  • HIVE-12566 - Incorrect result returns when using COALESCE in WHERE condition with LEFT JOIN
  • HIVE-12713 - Miscellaneous improvements in driver compile and execute logging
  • HIVE-12784 - Group by SemanticException: Invalid column reference
  • HIVE-12790 - Metastore connection leaks in HiveServer2
  • HIVE-12795 - Vectorized execution causes ClassCastException
  • HIVE-12946 - alter table should also add default scheme and authority for the location similar to create table
  • HIVE-13039 - BETWEEN predicate is not functioning correctly with predicate pushdown on Parquet table
  • HIVE-13065 - Hive throws NPE when writing map type data to a HBase backed table
  • HUE-3106 - [filebrowser] Add support for full paths in zip file uploads
  • HUE-3110 - [oozie] Fix bundle submission when coordinator points to multiple bundles
  • HUE-3180 - [useradmin] Override duplicate username validation message
  • IMPALA-1702 - Check for duplicate table IDs at the end of analysis (issue not entirely fixed, but now fails gracefully)
  • IMPALA-2264 - Implicit casts to integers from decimals with higher precision sometimes allowed
  • IMPALA-2473 - Excessive memory usage by scan nodes
  • IMPALA-2621 - Fix flaky UNIX_TIMESTAMP() test
  • IMPALA-2643 - Nested inline view produces incorrect result when referencing the same column implicitly
  • IMPALA-2765 - AnalysisException: operands of type BOOLEAN and TIMESTAMP are not comparable when OUTER JOIN with CASE statement
  • IMPALA-2798 - After adding a column to avro table, Impala returns weird result if codegen is enabled.
  • IMPALA-2861 - Fix flaky scanner test added via IMPALA-2473 backport
  • IMPALA-2914 - Hit DCHECK Check failed: HasDateOrTime()
  • IMPALA-3034 - MemTracker leak on PHJ failure to spill
  • IMPALA-3085 - DataSinks' MemTrackers need to unregister themselves from parent
  • IMPALA-3093 - ReopenClient() could NULL out 'client_key' causing a crash
  • IMPALA-3095 - Allow additional Kerberos users to be authorized to access internal APIs
  • KITE-1114 - fix test
  • KITE-1114 - Fix missing license header
  • KITE-1114 - Kite CLI json-import HDFS temp file path not multiuser safe
  • OOZIE-2413 - Kerberos credentials can expire if the KDC is slow to respond
  • OOZIE-2428 - TestSLAService, TestSLAEventGeneration flaky tests
  • OOZIE-2432 - TestPurgeXCommand fails
  • OOZIE-2435 - TestCoordChangeXCommand is flaky
  • SENTRY-835 - Drop table leaves a connection open when using MetastoreListener
  • SENTRY-885 - DB name should be case insensitive in HDFS sync plugin
  • SENTRY-944 - Setting HDFS rules on Sentry managed hdfs paths should not affect original hdfs rules
  • SENTRY-953 - External Partitions which are referenced by more than one table can cause some unexpected behavior with Sentry HDFS sync
  • SENTRY-957 - Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting up
  • SENTRY-988 - It's better to let SentryAuthorization setter path always fall through and update HDFS
  • SENTRY-991 - backportRoles of Sentry Permission needs to be case insensitive
  • SENTRY-994 - SentryAuthorizationInfoX should override isSentryManaged
  • SENTRY-1002 - PathsUpdate.parsePath(path) will throw an NPE when parsing relative paths
  • SENTRY-1003 - backportSupport "reload" by updating the classpath of Sentry function aux jar path during run time
  • SENTRY-1008 - Path should be not be updated if the create/drop table/partition event fails
  • SENTRY-1044 - Tables with non-HDFS locations breaks HMS startup
  • SOLR-7281 - Add an overseer action to publish an entire node as 'down'
  • SOLR-8367 - Fix the LeaderInitiatedRecovery 'all replicas participate' fail-safe
  • SOLR-8371 - Try and prevent too many recovery requests from stacking up and clean up some faulty cancel recovery logic
  • SOLR-8372 - backportCanceled recovery can lead to data loss
  • SOLR-8575 - Addendum to Fix HDFSLogReader replay
  • SOLR-8575 - Fix HDFSLogReader replay status numbers and a performance bug where we can reopen FSDataInputStream too often
  • SOLR-8615 - Just like creating cores, we should use multiple threads when closing cores
  • SOLR-8720 - ZkController#publishAndWaitForDownStates should use #publishNodeAsDown
  • SOLR-8771 - Multithreaded core shutdown creates executor per core
  • SQOOP-2847 - Sqoop --incremental + missing parent --target-dir reports success with no data
  • SQOOP-2422 - Sqoop2: Test TestJSONIntermediateDataFormat is failing on JDK8
  • ZOOKEEPER-442 - Need a way to remove watches that are no longer of interest"

 

Selected tab: WhatsNew

Want to Get Involved or Learn More?

Check out our other resources

Cloudera Community

Collaborate with your peers, industry experts, and Clouderans to make the most of your investment in Hadoop.

Cloudera University

Receive expert Hadoop training through Cloudera University, the industry's only truly dynamic Hadoop training curriculum that’s updated regularly to reflect the state of the art in big data.