New Features and Changes in Cloudera Altus Director

The following sections describe the new features and changes introduced in this release of Altus Director.

What's New in Altus Director 6.0

Altus Director 6.0 provides new features and supports new versions of CDH and Cloudera Manager. It also supports installation of clusters on additional operating system versions.

Cloudera Director Renamed Cloudera Altus Director

Cloudera Director is now part of the Cloudera Altus platform. The full name of Cloudera Director is now Cloudera Altus Director.

The name change does not affect scripts, commands, or file names that use the short name Director or Cloudera.

Support for Cloudera Manager 6.0 and CDH 6.0

By default, Altus Director 6.0 installs Cloudera Manager and CDH 6.0.

Altus Director 6.0 installs Java 8 with Cloudera Manager 6.0 and the CDH 6.0 instances managed by the Cloudera Manager 6.0 instance.

You can use Altus Director to install Cloudera Manager and CDH 5.7 or later versions if you specify the product version and parcel repositories in the Altus Director configuration file. When Altus Director 6.0 installs Cloudera Manager 5.7 or a later version, Altus Director 6.0 installs Java 7 with Cloudera Manager and the CDH instances managed by the Cloudera Manager.

Operating System Support for Altus Director

You can install Altus Director on RHEL/CentOS 7.5 or Ubuntu 16.04.

In addition, Altus Director continues to support the following OS versions:
  • CentOS and RHEL, versions 6.7, 6.8, 7.2, 7.4
  • Ubuntu 14.04

Support for Java 8

Altus Director 6.0 requires Java 8.

Altus Director 6.0 does not support Java 7 or earlier versions. It also does not support Java 9 or later versions.

Support for Cloudera Manager 5.7 or Higher

Altus Director 6.0 supports Cloudera Manager version 5.7 or higher.

Altus Director 6.0 does not work with Cloudera Manager instances earlier than version 5.7. Altus Director 6.0 will not start if it detects that a deployment running a version of Cloudera Manager earlier than 5.7

Operating System Support for Cloudera Manager

Altus Director 6.0 supports installation of Cloudera Manager and clusters on the following OS versions:
  • For AWS and Google Cloud Platform only, Altus Director 6.0 now supports RHEL and CentOS version 7.5
  • For all cloud service providers, Altus Director 6.0 continues to support RHEL and CentOS versions 6.7, 6.8, 7.2, and 7.4

Encryption of application.properties File Content

You can now encrypt the values of the properties in the Altus Director application.properties file using a master password.

Additional Script Model

As of Altus Director 6.0, when you create bootstrap, post-create, or pre-terminate scripts, you can provide each script as an object consisting of an ID string and the script content.

For example:
bootstrapScripts: [
  {
    id: "SCRIPT-01"
    content: """#!/bin/bash …
  }
]

You can use the new script model in addition to the previous model where you provide the script as an ordinary string.

Changes in Altus Director 6.0

Altus Director 6.0 has a number of changes that make it incompatible with the previous version. Review the following list of changes in Altus Director 6.0 before you upgrade to this version. The changes might require you to update your configuration files or modify the way that you use the product.

Support for New Azure VM Image

Altus Director 6.0 uses the cloudera-centos-67-latest VM image for Azure clusters.

If you have existing clusters in Azure that you need to grow or if you have an operation where you need to use an existing template that uses the cloudera-centos-6-latest image, you can add the image definition to the images.conf file so you can continue to use the cloudera-centos-6-latest image.

Add the following VM image definition to the images.conf file:
cloudera-centos-6-latest {
     publisher: cloudera
     offer: cloudera-centos-os
     sku: 6_7
     version: latest
     }

Restart Altus Director 6.0 to load the new image definition.

Custom images.conf files are stored in the Azure plug-in directory. If you create a custom images.conf file in Altus Director 6.0, add the file to the following directory: /var/lib/cloudera-director-plugins/azure-provider-3.0.0/etc/

If you use other versions of the Azure plug-in for Director, update the images.conf file in the appropriate plug-in directory.

Storage Accounts in Azure Supported Only with CLI

As of Altus Director 6.0, all new instance templates for Azure use Managed Disks. On the Altus Director 6.0 web UI, you can no longer create instance templates that use storage accounts. To create a cluster in Azure that uses storage accounts, you must use the Altus Director CLI.

You can use the Altus Director CLI to create clusters with storage accounts in one of the following ways:
  • Using an existing instance template that uses storage accounts.

    Use an unedited instance template created in previous versions of Altus Director to create clusters with storage accounts. Editing an instance template that uses storage accounts can change the settings of the template. For more information, see Edited or cloned Altus Director 2.8 instance templates for Azure do not work as expected.

  • Modifying the CLI configuration file to use storage accounts instead of managed disks in the instance template created in Altus Director 6.0.

    You can modify the configuration file that you use with the CLI to use storage accounts instead of managed disks. In the Instance Template section of the configuration file, set the managedDisks field to No.

    For more information about the managedDisks field, search for managedDisks in the reference configuration file for Azure.

Stand-alone CLI Commands Removed

As of Altus Director 6.0, the stand-alone commands are removed from the Altus Director client. The Altus Director client is now used only for command line interactions with the Altus Director server. If an Altus Director server is not available, you cannot run the Altus Director CLI commands.

The following list shows the CLI commands that have been removed and the commands that you can use instead:
Obsolete Command Replacement Command
bootstrap bootstrap-remote
status No replacement command. Use the server API or client libraries instead.
terminate terminate-remote
update No replacement command. Use the server API or client libraries instead.
validate validate-remote

For more information about the Altus Director command line interface, see Command Line Interface.

Default H2 Database for Stand-alone CLI Not Created

Altus Director 6.0 does not create the H2 database used to maintain the local state of the stand-alone client. With the stand-alone CLI commands removed, Altus Director 6.0 no longer requires the H2 database.

Server API Versions v1 to v5 Not Available

Altus Director 6.0 introduces version d6.0 of the Altus Director server API. Along with server API d6.0, Altus Director 6.0 supports server API versions v6 through v12. It no longer supports Director server API version v5 or earlier.

If you use an API version v5 or earlier, you must update your applications to use API v6 or later.

Changes to Configuration Properties in application.properties

The following configuration properties in the application.properties file have been renamed in Altus Director 6.0:
Old property name New property name
lp.terminate.deployment.clouderaManagerServerStopWaitTimeInMs lp.cleanup.cm.stopWaitTimeInMs
lp.terminate.deployment.timeBetweenConnectionRetriesInMs lp.cleanup.cm.timeBetweenConnectionRetriesInMs

If you customized the renamed properties in previous versions, you must rename the properties in the application.properties for Altus Director 6.0. Altus Director 6.0 ignores properties in application.properties file that use obsolete names.

Properties Removed from Cluster Configuration Files

When you the CLI to launch a cluster, you can use a configuration file to customize the cluster setup. Altus Director provides sample configuration files that you can for your own use.

When you the CLI to launch a cluster, you can use a configuration file to customize the cluster setup. In Altus Director 6.0, a number of properties have been removed. If you use these properties in your cluster configuration file, update the file to use the correct property names. If you leave the property in the cluster configuration file, Altus Director ignores the property.

For more information about the cluster configuration file, see The Altus Director Configuration File.

As of Altus Director 6.0, the following properties in the cluster configuration file have been removed and replaced with another property:
Template Removed Property Replacement
Deployment customBannerText Use the custom_banner_html property of the CLOUDERA_MANAGER service.
Instance bootstrapScript bootstrapScripts

Note the new property name is plural, ending in s.

Instance bootstrapScriptPath bootstrapScriptsPaths

Note the new property name is plural, ending in s.

Kerberos Must be Configured Using Configuration Files

As of Altus Director 6.0, you cannot enable Kerberos for a new Cloudera Manager deployment on the Altus Director web UI. The option for enabling Kerberos for a new Cloudera Manager deployment on the web UI is removed.

You must use a configuration file to enable Kerberos.

Changes to the Altus Director SDK Library

As of Altus Director 6.0, the Java and Python client libraries released with Altus Director are generated using a new framework. The new framework generated changes in the client libraries. For example:
  • In the Java client, enumerated values formerly represented by strings are now represented as Enum objects.
  • In the Python client, API method names and model field names now use snake case (get_status) instead of camel case (getStatus).
  • In the Python client, API exceptions are raised as APIException objects instead of HTTPError objects.

Code that uses Altus Director 2.x libraries does not work with the client libraries released with Altus Director 6.0. To use the new client libraries, you must update your code to match the changes in the client libraries.

You can continue to use Altus Director 2.x client libraries with Altus Director 6.0 through API versions v6 to v12. You need to update your code only if you want the code to work with the new client libraries released with Altus Director 6.0.

Changes to CDH 6.0 that affect Altus Director

CDH 6.0 has changes that affect Altus Director usage. The changes are not part of Altus Director, but they affect use of Altus Director.

External Spark 2 Parcels Not Supported in CDH 6.0
Prior to CDH 6.0, the use of Spark 2 requires the use of a separate parcel. This is no longer necessary in CDH 6.0, which includes Spark 2.2. However, use of a different version of Spark 2 is not supported with CDH 6.0 through the use of an external parcel. Existing Altus Director configuration files that bring in Spark 2 through a parcel must be modified for CDH 6.0 to rely on Spark 2.2 as packaged with CDH.
Sqoop 2 Not Supported in CDH 6.0

Sqoop 2 is not packaged with CDH 6.0. Bootstrapping CDH 6.0 clusters that call for Sqoop 2 will fail under Altus Director 6.0. Instead, use Sqoop 1 (service "SQOOP_CLIENT" and role "GATEWAY").

Sqoop 2 is still supported for CDH 5.x clusters, but deprecated.

Hue Service in CDH 6.0 Requires Python 2.7 and a Psycopg adapter

The Hue service included in CDH 6.0 requires a Python version that may not be available in the operating systems. The Hue service also requires a Psycopg adapter version that is different from the version used by the Cloudera Manager 6.0 agent package.

Altus Director provides a bootstrap script that you can use to install the correct version of Python and Psycopg adapter library for Hue. For more information about the bootstrap script, see Hue service in CDH 6 that uses the wrong version of Python fails to start.