New Features and Changes in Cloudera Manager 6.3.0

OpenJDK 11 support for Cloudera Manager and CDH 6.3 and higher

You can now use OpenJDK 11 with Cloudera Enterprise 6.3.

When you install OpenJDK 11 in your cluster, it uses the G1GC method for garbage collection for most services, which may require tuning to avoid overcommitting memory. See Tuning JVM Garbage Collection.

OPSAPS-50993, OPSAPSD-49390, OPSAPS-51643

Cloudera Manager Kerberos support for FreeIPA and Red Hat Identity Manager

Cloudera Manager now supports FreeIPA and RedHat Identity Manager as a Kerberos KDC type. See Step 4: Enabling Kerberos Using the Wizard

Cloudera Issue: OPSAPS-45833, OPSAPS-51348

SPNEGO/Kerberos support for Cloudera Manager Admin Console and API

The Cloudera Manager Admin Console and API now support Kerberos authentication (using SPNEGO). Only clients that can use SPNEGO (such as curl) are supported. The Swagger and Python SDKs for Cloudera Manager do not support SPNEGO.

With SPNEGO enabled, the Swagger-based Java and Python SDKs, as well as the older deprecated Java SDK, can still authenticate using HTTP Basic Authentication. The older deprecated Python SDK cannot. Do not enable SPNEGO if you are relying on the deprecated Python client for any operations.

You can enable SPNEGO/Kerberos authentication for the Cloudera Manager Admin Console and API by going to Settings > Administration > External Authentication > "Enable SPNEGO/Kerberos Authentication for the Admin Console and API" and checking the box. This requires a restart of Cloudera Manager to take effect. It also requires that Kerberos is enabled.

For more information, see Configuring External Authentication and Authorization for Cloudera Manager.

Cloudera Issue: OPSAPS-49753, OPSAPS-24297

Cloudera Manager Server certificate expiration monitoring and alerting

Cloudera Manager now alerts you 60 days before the Cloudera Manager Server TLS certificate expires. You can view and modify the threshold values by searching for Expiry in the Cloudera Management Service configuration page.

Cloudera Issue: OPSAPS-35977

Auto-TLS Support for Root CA

For new cluster installations, auto-TLS can create and use an intermediate CA for an existing internal root CA. This creates a chain of trust to your existing internal root CA, and allows internal hosts that trust your root CA to access cluster web resources without browser security warnings.

This feature is not supported for existing clusters.

Cloudera Issue: OPSAPS-50063

Service and Host Monitor configurations for Garbage Collection

The Cloudera Manager Host monitor and Service Monitor now use G1GC garbage collection by default. You can change this by adding Java options in the Java Configuration Options for Host Monitor and Java Configuration Options for Service Monitor configuration properties (Go to Clusters > Cloudera Management Service and search for "java". See Tuning JVM Garbage Collection.

Cloudera Issue: OPSAPS-50123

New Configuration Parameters for JournalNode Syncer

Enable JournalNode Syncer has been added for CDH 6.3 and later. When enabled, a JournalNode will periodically synchronize edit logs with other JournalNodes.

Shared Edits Directory has been added for HDFS JournalNodes when High Availability is enabled, which allows the JournalNode Syncer to start and work correctly.

Cloudera Issue: OPSAPS-51249

New Kafka configuration parameters and metrics

New Kafka configurations and default values are now available in Cloudera Manager when using the CDK 4 Kafka distribution:
  • num.network.threads=8
  • num.recovery.threads.per.data.dir=1
  • num.replica.fetchers=4 (default changed from 1)
New metrics:
  • broker
  • broker level topic
  • controller
  • fetch session cache
  • log cleaner
  • network
  • replica manager
  • session expire listener
  • zookeeper client

Cloudera Issue: OPSAPS-49741

Simplified enabling Kerberos through Cloudera Manager templates

Added a new section called enableKerberos to the instantiator section of Cloudera Manager templates. For examples:
"instantiator": { 
    "clusterName": "test", 

     "enableKerberos": { 
         "datanodeTransceiverPort" : <optional/default 1004>, 
         "datanodeWebPort" : <optional/default 1006> 
      }, 
      . 
      . 
This enables Kerberos on all the services in the template.

Cloudera Issue: OPSAPS-49704

Cluster Administrator role can now create and delete Data Context

Users with the Cluster Administrator role who can access only the base clusters in Virtual Private Cloud setups can now create and delete Data Contexts.

Cloudera Issue: OPSAPS-49478

New YARN Configuration Parameter

A new YARN configuration parameter, Enable Container Launch Debug Information, has been added and is disabled by default. Previously, this configuration had to be made using the NodeManager Advanced Configuration Snippet for yarn-site.xml.

Cloudera Issue: OPSAPS-50709

Kudu integration with Sentry

Cloudera Manager can now configure Kudu integration with Sentry.

Cloudera Issue: OPSAPS-50359

Network Performance Inspector now includes a bandwidth test

You can now test the bandwidth between clusters using the Network Performance Inspector. See Inspecting Network Performance

Cloudera Issue: OPSAPS-49682

Kafka support in Compute Clusters

You can now add the Kafka service to a compute cluster. Kafka will share the Sentry service across multiple Kafka instances.

Cloudera Issue: OPSAPS-49165

Improved Backup and Disaster Log Retention behaviour

When you are using a custom log location for BDR replication, automatic log expiration now purges these custom replication logs and metadata files, based on their purge settings. To maintain information for each replication schedule, it is important that user set valid paths for HDFS that are writable by an existing user.

OPSAPS-50880

Filtered HDFS NameNode and Hive MetaStore role logs included with BDR diagnostic bundles

Filtered HDFS NameNode and Hive MetaStore role logs originating from both source and target clusters are now included in replication diagnostic bundles. The logs can be filtered based on these criteria:
  • Earliest start and latest end timestamps for all the replication runs within a diagnostic bundle
  • The configured log level (defaults to INFO) on both the source and destination clusters for HDFS and Hive replication jobs.
The following role logs are included:
  • HDFS NameNode (for HDFS and Hive replication jobs)
  • Hive MetaStore (Hive replication)

Both source and destination role logs will be capped to a maximum of 201 MBs (this is the normal CDH role log size).

You configure these log collections using the following Advanced configuration snippets:
  • HDFS Replication Advanced Configuration Snippet (Safety Valve) for hdfs-site.xml
  • Hive Replication Advanced Configuration Snippet (Safety Valve) for hive-site.xml
Enter the following configuration parameters in the appropriate Advance Configuration Snippet for Hive or HDFS (default values are in brackets):
  • SKIP_HIVE_LOG_COLLECTION [false]
  • SKIP_HDFS_LOG_COLLECITON [false]
  • LOG_LEVEL [INFO]
  • LOG_SEARCH_TIMEOUT_MILLIS [300000]
  • LOG_ROLE_RESULT_LIMIT [10000]
  • LOG_TOTAL_BYTES_LIMIT [210763776]

Cloudera Issue: OPSAPS-49988

Option to disable metric rollup

You can now disable rollup of metrics over time. This can help increase performance of the Service Monitor. See Disabling Metric rollup.

Cloudera Issue: OPSAPS-50149