Enabling TLS with Altus Director

Transport Layer Security (TLS) is a security protocol that supersedes Secure Sockets Layer (SSL). It is designed to prevent eavesdropping, tampering, and message forgery by encrypting network communications. It also supports authentication of host certificates prior to encryption, to prevent spoofing. You can enable TLS on your clusters, as well as on Cloudera Manager and Altus Director, in order to protect communications among them.

You can enable TLS for the following components:
  • Cloudera Manager and CDH
  • Altus Director server and client
  • Altus Director database

Although configuring TLS for the different components are separate and independent tasks, a comprehensive security plan could require that you configure TLS for all components.

Enabling TLS for Cloudera Manager and CDH

You can enable TLS for use the Cloudera Manager and CDH by using auto-TLS or by manually setting up TLS. Altus Director can also work with an existing Cloudera Manager installation over TLS.

Automatic TLS (Auto-TLS)

Altus Director 2.6 and higher can work in concert with Cloudera Manager 5.13 and higher to automatically configure TLS for Cloudera Manager and CDH. This is called auto-TLS. Auto-TLS replaces the work otherwise performed manually to create key pairs and public key certificates, copy them into the correct locations for Cloudera Manager and CDH, and alter Cloudera Manager and CDH component configurations to use them and support access over TLS. Auto-TLS configures TLS as soon as possible, often before relevant services are started, so there is no span of time when services are listening for unencrypted traffic.

Auto-TLS creates a bespoke certificate authority (CA) on the Cloudera Manager server instance. All certificates generated by auto-TLS are signed by this CA. Since the CA does not exist before Cloudera Manager is installed, and because the root certificate for the CA is not itself signed by a pre-existing CA, it is difficult to establish trust for the certificates ahead of time. If you wish to have certificates signed by a pre-existing CA instead of the bespoke CA, then continue to set up TLS manually, and follow the instructions in Manual TLS to configure Altus Director to communicate with Cloudera Manager over TLS.

Using Auto-TLS

To use auto-TLS, you must either use an Altus Director configuration file or Altus Director's REST API for bootstrapping Cloudera Manager. Auto-TLS is not available through the Altus Director UI.

To enable auto-TLS for a Cloudera Manager deployment using a configuration file, simply include the tlsEnabled property in the cloudera-manager section, set to true.
...
cloudera-manager {

    tlsEnabled: true

...
}
...
To enable auto-TLS for a Cloudera Manager deployment using the Altus Director REST API, simply set the tlsEnabled property of the deployment template to true. You must use at least version 10 of the API. An example JSON snippet is below.
...
  "tlsEnabled": true,
...

When auto-TLS is enabled for a Cloudera Manager deployment, every CDH cluster managed by that Cloudera Manager installation is also automatically configured for TLS. If it is necessary for a CDH cluster to not be configured for TLS while Cloudera Manager is, then follow the instructions for Manual TLS to configure Cloudera Manager instead.

Use of auto-TLS causes the automatic installation of unlimited strength JCE policy files on the Cloudera Manager instance by Altus Director. The policy files eliminate runtime restrictions on the strength of cryptographic algorithms that are included in Java.

Altus Director automatically retrieves the root certificate of the bespoke CA used in auto-TLS from the Cloudera Manager instance, and establishes trust in it. This creates an opportunity for a man-in-the-middle attack on the SSH connection Altus Director uses for the retrieval. For protection against such an attack, consider configuring SSH host key retrieval and verification.

Retrieving the CA Root Certificate

When a browser or other web client accesses a Cloudera Manager instance set up under auto-TLS, the browser or web client should emit a warning that the server certificate for Cloudera Manager is not trusted. This is because the certificate is signed by the bespoke CA resident on the Cloudera Manager instance, and that CA is not trusted by any of the well-known default CAs installed in the client. It is possible to override or ignore these warnings, but a more secure option is to import the root certificate of the CA into the client.

There are a few ways to retrieve the root certificate of the bespoke CA, so that it can be imported into the trusted certificate store of a client. One easy way is to use the Altus Director REST API, or an API client library, to get the deployment corresponding to the Cloudera Manager instance. The result data includes the root certificate as the trustedCertificate field. The value of the field is a single line containing the entire certificate contents; to use the value as a proper encoded certificate, replace each continuous sequence of \r and \n strings with a single newline.

http://director_host:7189/api/v10/environments/env_name/deployments/dep_name

{
  "name": "dep_name",
  "hostname": "203.0.113.101",
  "port": 7183,
  ...
  "tlsEnabled": true,
  "trustedCertificate": "-----BEGIN CERTIFICATE-----\r\nMII...\r\n-----END CERTIFICATE-----\r\n",
  ...
}

Once the root certificate is imported for a client, it should be trusted, and no connection warnings should appear.

Advanced Auto-TLS Configuration

Auto-TLS requires no detailed configuration in order to work properly. However, there are a set of TLS configuration properties that can be passed through an Altus Director deployment template to Cloudera Manager. The properties are listed below; each property's key and value is a string.
TLS Configuration Properties
Property Name Purpose Default Value
trusted_ca_certs path to a file containing a concatenation of certificates to be trusted by cluster services none; see below
subject_suffix suffix for DNs generated for auto-TLS "ST=CA,C=US"
ca_dn complete DN for CA root certificate (overrides ca_name and subject_suffix) none
ca_name CN for subject DN of CA root certificate, used in concert with subject_suffix; cannot exceed 64 characters "SCM Local CA on deployment deployment-name", truncated if necessary to 64 characters
email_address email address to include as subject alternative name (SAN) on all certificates none
ca_key_algo algorithm used to generate CA key: RSA, DSA, or EC RSA
ca_key_args arguments for CA key generation: for RSA and DSA, key length; for EC, curve name 3072
ca_sig_hash_algo hash algorithm for CA key signature: SHA256, SHA512 SHA256
host_key_algo algorithm used to generate host keys: RSA, DSA, or EC RSA
host_key_args arguments for host key generation: for RSA and DSA, key length; for EC, curve name 3072
host_sig_hash_algo hash algorithm for host key signatures: SHA256, SHA512 SHA256
ca_expiration date of CA certificate expiration, in YYMMDD format about one year from current time
host_expiration date of host certificate expirations, in YYMMDD format about one year from current time
key_encryption_algo encryption used for private keys stored on Cloudera Manager instance: AES128, or AES256 AES256
keytool path to Java keytool on Cloudera Manager instance see below

The trusted_ca_certs TLS configuration property is useful when cluster services need to connect over TLS to external endpoints, and the server certificates for those endpoints are not ultimately signed by widely acknowledged CAs. The self-signed server or CA certificates for those endpoints, in PEM format, should be included in the file indicated by the property. Since the property value is a path to a file containing the certificates, and not the certificates themselves, measures must be taken to ensure that the file is in place at the indicated path on the Cloudera Manager instance; for example, the file could be included in the instance's base image, or the file could be written through a bootstrap script.

The host_expiration and ca_expiration TLS configuration properties set the expiration dates for server certificates issued for cluster services and for the Cloudera Manager server, respectively. They default to a date about one year in the future from the time when the certificates are issued. Beyond the expiration dates, the certificates are not accepted by TLS clients. Auto-TLS does not support the renewal of automatically generated certificates, so for long-lived clusters, specify expiration dates further in the future.

If the keytool TLS configuration property is passed to Altus Director, then its value is used as is. Otherwise, Altus Director attempts to locate keytool on the running Cloudera Manager instance. If Altus Director succeeds in locating keytool, it passes the path to keytool to Cloudera Manager; otherwise, it passes a default of keytool, which works if keytool is on the default PATH.

To set a TLS configuration property using a configuration file, include its key and desired value in a simple object named tlsConfigurationProperties in the cloudera-manager section. If no TLS configuration properties need to be set, the tlsConfigurationProperties object can be omitted.
...
cloudera-manager {

    tlsEnabled: true
    tlsConfigurationProperties {
        ca_key_args: 4096
        host_key_args: 4096
    }

...
}
...
To set a TLS configuration property using the Altus Director REST API, include it in the tlsConfigurationProperties map property of the deployment template. An example JSON snippet is below. If no TLS configuration properties need to be set, the tlsConfigurationProperties map property can be omitted or left empty.
...
  "tlsEnabled": true,
  "tlsConfigurationProperties": {
    "ca_key_args": "4096",
    "host_key_args": "4096"
  },
...

Altus Director does not attempt to validate TLS configuration properties, so any unknown or invalid properties are passed through as-is to Cloudera Manager. Check Altus Director and Cloudera Manager logs if the configuration properties appear to cause problems or have no effect.

Name Length Restrictions

Some standard naming restrictions are relevant when configuring TLS.
  • The fully-qualified domain name (FQDN) of a Linux instance can be up to 255 characters in length.
  • The value of the common name (CN) attribute in a distinguished name (DN) cannot exceed 64 characters.

Because auto-TLS normally uses the FQDN of an instance as the value of the CN attribute in that instance's certificate's subject DN, it is possible for certificate generation to fail if that FQDN is too long. When the FQDN exceeds 64 characters, a certificate with the FQDN as the value for the CN attribute cannot be generated.

This is of particular importance when using Google Compute Engine, which includes an instance's Altus Director virtual instance ID, a 36-character UUID, in the instance's hostname. Google Compute Engine also uses the project name as part of the domain name, consuming more characters from the allowed total of 64.

Therefore, when using auto-TLS, you might need to specify a short instance name prefix for Altus Director instance templates in order to stay below the 64-character limit. In the Altus Director UI, the instance name prefix is specified under the Advanced Options when creating or editing an instance template. In an Altus Director configuration file, the instance name prefix can be specified in the "provider" section when using either AWS, Google Cloud Platform, or Microsoft Azure as a cloud provider.
provider {
  instanceNamePrefix: abc
  ...
}

Static Private IP Addresses

In order to be compatible with Altus Director, the server certificate for the Cloudera Manager instance must include its private IP address as a subject alternative name (SAN). If the instance changes its private IP address, perhaps by being stopped and started again, then Altus Director will not be able to communicate with Cloudera Manager over TLS due to the mismatch.

There is currently no mechanism in auto-TLS to accommodate a change in the private IP address of the Cloudera Manager instance. Therefore, it is essential that the private IP address for the Cloudera Manager instance be assigned until the instance is terminated, and not released when it is stopped.

This is the normal behavior for instances under AWS and Google Cloud Platform. However, the default behavior for Microsoft Azure is to use dynamic private IP address allocation, which can cause the private IP address to change when the VM is stopped and then started again. In constrast, a static private IP address allocation remains with a VM until it is terminated. To change the private IP address allocation for an Azure VM from dynamic to static, use the Azure portal or Powershell. For more information about configuring Altus Director on Azure to use static IP addresses, see the first note in Notes on Pausing a Cluster on the page Pausing Altus Director Instances.

Coping with Untrusted Certificates Created by Auto-TLS

Server certificates signed by the bespoke CA are not trusted by web browsers or other clients, leading them to either produce warnings or refuse to complete TLS handshakes. There are several strategies for coping with the certificates.

First, clients can be configured to trust the root certificate of the CA by importing that certificate into the client's trusted certificate store. Once this is done, the client trusts any server certificates signed by the CA, just as it would for those signed by widely recognized CAs. However, this process must be performed for each new CA that is created by auto-TLS, and it usually requires manual steps.

A more user-friendly solution is to place a reverse proxy in front of Cloudera Manager and the cluster services. A reverse proxy can be configured to listen over TLS, using a server certificate signed by a recognized CA so that clients need not import any new trusted certificates. The server name for the proxy can be in your organization's domain and managed by your organization's CA, whether internal or third-party. In response to requests, the reverse proxy then can route new requests, again over TLS, to Cloudera Manager and cluster services.

For example, here is a configuration snippet for the nginx HTTP server and proxy that routes incoming TLS requests to the HDFS namenode web UI, also serving over TLS. In the example, hadoopproxy is the hostname of the instance hosting nginx, and nn.cloudprovider.internal refers to the instance hosting the HDFS namenode. A client request to https://hadoopproxy/ connects to the HDFS namenode web UI, even if the client cannot reach the HDFS namenode instance directly.
server {

  listen 443 ssl;
  server_name hadoopproxy;

  # public key certificate for nginx, signed by recognized CA
  ssl_certificate /etc/nginx/cert.pem;
  # private key for nginx
  ssl_certificate_key /etc/nginx/key.pem;

  location / {
    proxy_pass https://nn.cloudprovider.internal:50470/;
    # root certificate for bespoke CA on Cloudera Manager instance
    proxy_ssl_trusted_certificate /etc/nginx/cmca.crt;
    proxy_ssl_verify on;
    proxy_ssl_verify_depth 2;
    proxy_ssl_session_reuse on;
  }

}

Cloud provider load balancing services can also work as reverse proxies. For example, an application load balancer created through AWS Elastic Load Balancing (ELB) can be configured as a reverse proxy, listening for requests over TLS using a server certificate signed by a recognized CA and forwarding them over TLS to Cloudera Manager or cluster service web UIs. Check the documentation for your cloud provider for the available options.

Manual TLS

You might wish to set up TLS manually for Cloudera Manager and CDH instead of through the auto-TLS capability. One common reason for doing so is in order to use certificates signed by an existing CA, as opposed to the bespoke CA set up under auto-TLS. Altus Director supports this workflow by allowing you to enable TLS, via the Altus Director API, on a bootstrapped deployment.

The process of configuring TLS manually for Cloudera Manager and CDH is documented in Configuring Cloudera Manager Clusters for TLS/SSL, and the same process should be followed for installations bootstrapped by Altus Director. After the initial part of the process, Level 0, Altus Director must be updated to use TLS for communication. The remaining parts of the process should still be performed.

Configuring Level 0 for Cloudera Manager

Start by bootstrapping a deployment and cluster normally, without TLS enabled. Once the deployment and cluster are READY in Altus Director, perform the usual manual steps for configuring TLS in Cloudera Manager, with a slight modification explained below. See Level 0: Basic TLS/SSL Configuration for more information. The only steps that must be performed before updating Altus Director are those for Level 0, which reconfigure the Cloudera Manager server to use TLS. For complete Cloudera Manager TLS configuration, after updating Altus Director, be sure to continue the steps after Level 0 as usual for securing Cloudera Manager and CDH manually.

In order to be compatible with Altus Director, the server certificate for Cloudera Manager must include a subject alternative name (SAN) of type iPAddress with the private IP address of the Cloudera Manager instance. The stock instructions for configuring Level 0 do not include a SAN, but keytool allows specifying one using the -ext option. The example below illustrates how to use the -ext option with an IP address and a hostname; replace the sample IP address with the correct one when used. Because TLS clients might ignore the hostname in the common name (CN) attribute of the certificate's subject DN in lieu of SANs, you should include the hostname as a SAN as well.
$ sudo keytool -genkeypair -alias $(hostname -f)-server -keyalg RSA \
 -keystore /opt/cloudera/security/pki/$(hostname -f)-server.jks \
 -keysize 4096 -dname "CN=$(hostname -f),O=cloudera.com,ST=CA,C=US" \
 -ext "san=ip:203.0.113.101,dns:$(hostname -f)" \
 -storepass cloudera -keypass cloudera
During Level 0 configuration, you have the opportunity to have the public key certificate for the server signed by a CA. If desired, you can proceed with a self-signed certificate. To generate a self-signed certificate from the server key pair, use keytool. The self-signed certificate must be imported into the Java truststore (the file jssecacerts created in the Level 0 documentation), just as the root certificate for an internal CA would need to be. Be sure to perform this extra step after generating a self-signed certificate, instead of generating a certificate signing request (CSR).
$ sudo keytool -exportcert -rfc \
 -keystore /opt/cloudera/security/pki/$(hostname -f)-server.jks \
 -alias $(hostname -f)-server \
 -file /opt/cloudera/security/pki/$(hostname -f)-server.crt \
 -storepass cloudera -keypass cloudera

A self-signed certificate is untrusted by browser and other web clients, so there is often no reason to use one instead of taking advantage of auto-TLS.

Updating Altus Director

Any time after Level 0 configuration is complete, it is possible to update the deployment information in Altus Director to enable TLS communication. Two pieces of information are required:
  1. The encrypted port number for Cloudera Manager. This is usually 7183.
  2. Either the self-signed certificate for the Cloudera Manager server or, if using an internal CA, either the signed public key certificate for the server or the root certificate for the internal CA. No matter which certificate you choose, it is called the trusted certificate for the deployment.

Retrieve the deployment template for Cloudera Manager from Altus Director's REST API. You must use version 10 or higher of the Altus Director API. API access is available through the API console at http://director_host:7189/api-console/, or through an Altus Director API client library, or directly using your preferred REST client framework.

http://director_host:7189/api/v10/environments/env_name/deployments/dep_name/template

Edit the returned JSON to add or change the following fields in the top-level object:
  • tlsEnabled, with a value of true
  • port, with the port number as its numeric value, even if the default port of 7183 is to be used
  • trustedCertificate, with a string value matching the contents of the trusted certificate. Because the certificate is multiple lines, remove all newline characters except for one at the start of the key and one at the end of the key (as in the example below).
Here is an example of the new fields, with most of the contents of the trusted certificate omitted for brevity. The order and placement of the three fields isn't important, but they must be top-level fields in the JSON object:
"name": deployment_template_name  # this 'name' is already in your JSON template
"tlsEnabled": true,
"port": 7183,
"trustedCertificate": "-----BEGIN CERTIFICATE-----\nMII...\n-----END CERTIFICATE-----"

Finally, update the deployment by issuing an HTTP PUT to the deployment update endpoint. Include the entire modified deployment template in the request body, not just the new fields.

http://director_host:7189/api/v10/environments/env_name/deployments/dep_name

During the deployment update process, Altus Director attempts to connect to Cloudera Manager over TLS, using the port and trusted certificate supplied in the update. The update will fail if Altus Director cannot successfully connect.

Completing Manual TLS

After successful deployment update, Altus Director will communicate with Cloudera Manager exclusively over TLS. Be sure to return to the documented process of configuring TLS for other components of Cloudera Manager, such as its agents, to complete the work of securing Cloudera Manager and CDH clusters.

Disabling Manual TLS

If it becomes necessary to disable TLS for Cloudera Manager, it is possible to update Altus Director so that it communicates over unencrypted HTTP to Cloudera Manager. Performing such an update is only necessary if TLS is disabled for the Cloudera Manager API and web interface; if TLS is only disabled, say, for Cloudera Manager agent communication, then Altus Director does not need to be updated. In other words, Altus Director should be updated only if Level 0 is unconfigured for Cloudera Manager.

Also, if TLS is disabled only temporarily for Cloudera Manager, and then re-enabled soon afterward, then Altus Director will re-establish communication on its own. While TLS is disabled for Cloudera Manager, Altus Director will be unable to communicate with it. Often this is not a problem. However, health reports, refresh processes, and usage-based billing will not function while Altus Director is out of contact.

Disabling TLS for Cloudera Manager should not be necessary when auto-TLS is in use. These instructions assume that TLS was configured manually for Cloudera Manager.

Begin by retrieving the deployment template for Cloudera Manager from Altus Director's REST API. You must use version 10 or higher of the Altus Director API. API access is available through the API console at http://director_host:7189/api-console/, or through an Altus Director API client library, or directly using your preferred REST client framework.

http://director_host:7189/api/v10/environments/env_name/deployments/dep_name/template

Edit the returned JSON to add or change the following fields in the top-level object:
  • tlsEnabled, with a value of false
  • port, with the port number as its numeric value, even if the default port of 7180 is to be used
  • trustedCertificate, which should be omitted completely from the template
Here is an example of the updated fields:
"tlsEnabled": false,
 "port":7180

Finally, update the deployment by issuing an HTTP PUT to the deployment update endpoint. Include the entire modified deployment template in the request body, not just the updated fields.

http://director_host:7189/api/v10/environments/env_name/deployments/dep_name

During the deployment update process, Altus Director attempts to connect to Cloudera Manager over an unencrypted connection, using the port supplied in the update. The update will fail if Altus Director cannot successfully connect.

After successful deployment update, Altus Director will communicate with Cloudera Manager without TLS. If or when TLS is enabled once again for Cloudera Manager, repeat the procedure above for updating Altus Director to communicate over TLS.

Working with an Existing Cloudera Manager

Altus Director allows you to bootstrap a cluster using a deployment of Cloudera Manager that already exists, and was not itself created by Altus Director. When using an existing Cloudera Manager, auto-TLS is not available. However, Altus Director can still work with an existing Cloudera Manager installation over TLS.

If the existing Cloudera Manager installation is already configured for TLS, then the Altus Director configuration file should indicate that in the cloudera-manager section by:
  • setting the tlsEnabled property to true
  • optionally setting the port property to the encrypted port (or else the default of 7183 is used)
  • setting the trustedCertificate property to a string value matching the contents of the trusted certificate; because the certificate is multiple lines, replace each newline in it with a two-character \n string

If the existing Cloudera Manager installation is not configured for TLS when the cluster is bootstrapped, but TLS is then manually enabled later, then you should update the deployment information in Altus Director. See Updating Altus Director above for instructions on how to do so; the procedure is identical to that for a Cloudera Manager installation that was originally bootstrapped by Altus Director. As noted above, the server certificate for Cloudera Manager must include a subject alternative name (SAN) with the private IP address of the Cloudera Manager instance in order to be compatible with Altus Director.

Enabling TLS for the Altus Director Server and Client

You can configure TLS for Altus Director to secure access to the Altus Director server and its API endpoints. You can configure Altus Director for TLS before or after a Cloudera Manager deployment.

To configure Altus Director to use TLS, complete the following steps:
  1. Configure the keystore.
  2. Configure the truststore.
  3. Enable TLS on the client.

If you are using Altus Director in a non-production environment, you can generate a test keystore and configure TLS using a a self-signed certificate. For more information about generating a keystore and self-signed certificate, see Generating a Test Keystore for Altus Director.

If you need to use a hostname that is private to your cloud provider, you can stop the Altus Director client from verifying the hostname. For more information about disabling hostname verification, see Disabling Hostname Verification.

Configuring the Keystore

Altus Director is configured for TLS by setting its own configuration properties in its application.properties file. See Setting Altus Director Properties for information on where the file is located and details on how to set properties.
Property Description
server.ssl.key-store Path to keystore holding server's private key and public key certificate.
server.ssl.key-store-password Password for keystore.
server.ssl.key-store-type Type of keystore. Optional.
server.ssl.key-store-alias Alias of private key in keystore, for JKS type keystores. The default is to expect only a single alias in the keystore. Optional.
server.ssl.key-password Password for private key, if different from password for keystore. Optional.

Additional properties are supported. Consult the Spring Boot documentation on configuring SSL for details.

After setting TLS configuration properties, restart Altus Director for it to begin requiring TLS. Director continues to listen on the same port as before (default 7189) unless configured for a different port.

Once Altus Director is configured for TLS, access its web interface using https URLs. If the server certificate is not signed by a certificate authority recognized by the client (for example, the web browser), then take the appropriate steps to either add an exception for the server or add the server certificate, or one from a responsible certificate authority, to the client's truststore.

Configuring the Truststore

To use TLS with Altus Director, you must have a truststore for the certificates from trusted certificate authorities (CA) that are used to verify the server certificate. If the Altus Director server presents a certificate that is not signed by a CA whose root certificate is in the truststore, then the client is not able to connect to the server.

By default, the Altus Director client uses the Java truststore file cacerts for trusted certificates. If the Altus Director server certificate is not in the default Java truststore, you can add it to the truststore using a keytool command. You can also create a custom truststore for your Altus Director server certificate.

The following example command shows how you can create a custom truststore by copying the default Java truststore and adding a server certificate or certificate authority root certificate.
$ cp $JAVA_HOME/jre/lib/security/cacerts directorclientts.jks
$ keytool -importcert -alias director -file director.crt \
  -keystore directorclientts.jks -storepass changeit -noprompt
To use a custom truststore with the Altus Director client, set the appropriate Java SSL system properties using the DIRECTOR_CLIENT_JAVA_OPTS environment variable. At least the "javax.net.ssl.trustStore" property must be set. The contents of the environment variable are passed to the Altus Director client JVM when using the cloudera-director run script.
$ export DIRECTOR_CLIENT_JAVA_OPTS=\
"-Djavax.net.ssl.trustStore=/path/to/directorclientts.jks \
-Djavax.net.ssl.trustStorePassword=changeit"
$ cloudera-director bootstrap-remote ...

In some cases, when Altus Director performs internal API calls, such as API calls for collecting usage information, the Altus Director server requires a client connection.

To use a custom truststore with the Altus Director server, set configuration properties in its application.properties file. See Setting Altus Director Properties for information on where the file is located and details on how to set properties.

Property Description
server.ssl.trust-store Path to truststore holding trusted certificate for server, and other certificates.
server.ssl.trust-store-password Password for truststore.

After setting TLS configuration properties, restart Altus Director for them to take effect.

Enabling TLS on the Altus Director Client

When the Altus Director server is configured for TLS, remote commands issued to the Altus Director client, such as bootstrap-remote and terminate-remote, must indicate that TLS is enabled on the server. Do so by passing the following configuration properties to the client:
  • lp.remote.tlsEnabled, set to true
  • lp.remote.hostAndPort, set to the hostname used in the CN component of the subject DN of the server certificate, followed by a colon and the server port
Here is an example bootstrap-remote command that works with an Altus Director server that has TLS enabled.
$ cloudera-director bootstrap-remote myconfig.conf \
  --lp.remote.hostAndPort=$(hostname -f):7189 \
  --lp.remote.username=admin --lp.remote.password=admin \
  --lp.remote.tlsEnabled=true

Generating a Test Keystore for Altus Director

To generate a simple test keystore for Altus Director, use the keytool command. An example command is below.
$ keytool -genkeypair -alias director -keyalg RSA \
  -keystore director.jks \
  -keysize 4096 -dname "CN=$(hostname -f),O=cloudera.com,ST=CA,C=US" \
  -storepass cloudera -keypass cloudera

Use a valid hostname for the CN (common name) component of the subject DN (distinguished name) for the public key certificate. Doing so allows clients to perform hostname verification.

Then, configure Altus Director with this keystore to enable TLS. Note that since the public key certificate in the keystore is self-signed, clients such as web browsers will issue warnings that the certificate is not trusted.
server.ssl.key-store: /path/to/director.jks
server.ssl.key-store-password: cloudera
To extract the self-signed certificate from the test keystore, use keytool again, as in the following command. The resulting certificate file can be imported into a truststore using the keytool -importcert command.
$ keytool -exportcert -rfc \
  -keystore director.jks -alias director \
  -file director.crt -storepass cloudera -keypass cloudera

Avoid using a self-signed certificate for Altus Director in a production environment. Instead, use a certificate signed by a recognized certificate authority.

Disabling Hostname Verification

By default, the Altus Director client performs normal hostname verification based on the CN component of the subject DN. Hostname verification can be disabled by passing the lp.remote.hostnameVerificationEnabled configuration property to the client, set to false (the property's default value is true).

This is helpful, for example, when connecting to an Altus Director instance whose certificate specifies a hostname that is private to your cloud provider, but a different, public hostname must be used. (A superior resolution to this problem is to include the public hostname as a subject alternative name in Altus Director's server certificate.)
$ cloudera-director bootstrap-remote myconfig.conf \
  --lp.remote.hostAndPort=public-hostname.company.com:7189 \
  --lp.remote.username=admin --lp.remote.password=admin \
  --lp.remote.tlsEnabled=true \
  --lp.remote.hostnameVerificationEnabled=false

Enabling TLS for the Altus Director Database

By default, when the Altus Director server is configured to use a MySQL or Maria DB database for its own data, it communicates with the hosting MySQL or Maria DB server over an unencrypted connection. You can configure the Altus Director server to communicate with the MySQL or Maria DB server over TLS.
Setting up TLS for communication with MySQL or Maria DB adds a few steps to the general procedures described in Using MySQL for Altus Director Server and Using MariaDB for Altus Director Server. As part of configuring the MySQL server, include the configuration options needed for TLS support. Consult MySQL documentation for complete information, depending on the version of MySQL in use. Some example options:
[mysqld]
ssl-ca=ca.pem
ssl-cert=server-cert.pem
ssl-key=server-key.pem

In this example:

  • ca.pem is a file containing the certificate for the certificate authority (CA) that has signed the server certificate

  • server-cert.pem is a file containing the public key certificate for the server

  • server-key.pem is a file containing the private key for the server

Configure the set of lp.database configuration properties for the Altus Director server as usual to point to the MySQL or Maria DB database server and the Altus Director database created within it. Be sure to explicitly supply values for lp.database.host, lp.database.port, and lp.database.name, even if their defaults apply.

Then, set the value of the lp.database.url configuration property to the following (all on one line):
jdbc:mysql://${lp.database.host}:${lp.database.port}/${lp.database.name}?verifyServerCertificate=true&useSSL=true&requireSSL=true
Finally, import the CA certificate to the Java system truststore. This allows Altus Director to trust the server certificate used by the MySQL server. Use a unique, appropriate alias for the certificate in the truststore. Do this before starting Altus Director.
sudo keytool -importcert -keystore $JAVA_HOME/lib/security/cacerts \
 -file ca.pem -alias mysqlca

As an alternative to importing the certificate into the Java system truststore, you can import the certificate into a separate truststore that is passed to Altus Director via the javax.net.ssl.trustStore system property. If you do this, be sure the separate truststore contains all other necessary certificates for Altus Director to function, such as those providing the basis of trust for communicating with your chosen cloud provider, and either Altus Director's own server certificate or a CA certificate in its chain.

When the Altus Director server is started, it will communicate with the MySQL or Maria DB server over TLS. If TLS communications cannot be established, Altus Director will exit.

Altus Director might fail to start and report the error "Path does not chain with any of the trust anchors" in its log. This might occur, depending on the version of Java in use and characteristics of the CA root certificate. If this error appears, then import the server certificate for the MySQL or Maria DB server itself into the truststore and restart the Altus Director server.

Using an AWS RDS MySQL Server

You can use a MySQL server launched from AWS RDS to host Altus Director's database. RDS automatically configures each MySQL database instance to support TLS connections, generating an appropriate server certificate for the instance during launch time. The server certificate is signed by a CA maintained by RDS itself.

You must import a certificate from RDS into the Java truststore for Altus Director to be able to communicate with the RDS instance over TLS. RDS maintains a root CA that governs most of its regions. RDS also maintains intermediate CAs for each region. Either the root CA certificate or the intermediate CA certificate for the region where the MySQL server resides should work to establish trust from Altus Director to the RDS instance over TLS.

Download the RDS root CA certificate from the link available at SSL Support for MySQL DB Instances in the AWS documentation. Download the RDS intermediate CA certificate for the region where the MySQL server resides from the appropriate link available at Intermediate Certificates in the AWS documentation.

If using the root CA certificate results in the error "Path does not chain with any of the trust anchors," then switch to using the intermediate CA certificate.