Troubleshooting TLS/SSL Issues in Cloudera Manager
This topic contains instructions for diagnosing and fixing issues you might face on a TLS-enabled cluster.
Inspecting Cloudera Manager Connectivity with OpenSSL
The openssl tool can be run from the host that is running the Cloudera Manager Agent or client service that should be inspected for connectivity issues. You should also test whether the certificate in use by the host is recognized by a trusted CA during the TLS/SSL negotiation.
$ openssl s_client -connect [host.fqdn.name]:[port]For example:
$ openssl s_client -connect test1.sec.cloudera.com:7183A return code 0 means openssl was able to establish trust of the server through its library of trusted public CAs. If the certificate was self-signed (recommended only on test clusters) or provided by a private CA, it might be necessary to add the private CA or self-signed certificate to the truststore using the openssl command. Adding the path to the root CA, -CAfile </path/to/root-ca.pem>, allows openssl to verify your self-signed or private CA-signed certificate as follows:
$ openssl s_client -connect test1.sec.cloudera.com:7183 -CAfile \ /opt/cloudera/security/CAcerts/RootCA.pem
... Verify return code: 0 (ok) ---
2014-10-06 21:33:47,515 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.2 2014-10-06 21:33:47,572 INFO WebServerImpl:org.mortbay.log: Started SslSelectChannelConnector@0.0.0.0:7183 2014-10-06 21:33:47,573 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180 2014-10-06 21:33:47,573 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
Uploading Diagnostic Bundles to Cloudera Fails
By default, Cloudera Manager uses HTTPS to upload diagnostic bundles to the Cloudera Support server at cops.cloudera.com. These uploads will fail if Cloudera Manager cannot confirm the authenticity of the Cloudera Support server. The Support server authenticates itself to Cloudera Manager by presenting a certificate signed by a public Certificate Authority (CA). However, if you have previously enabled Level 3 TLS Authentication for Cloudera Manager, the Cloudera Manager truststore may not contain certificates signed by public CAs. Therefore, attempting to verify the authenticity of the Support server's certificate will result in an authentication failure, and diagnostic bundle uploads will fail.
- Option A - Obtain the Support server's certificate and explicitly import it into your
- Option B - Use Java's default collection of public CA certificates, cacerts or jssecacerts, located at [JAVA_HOME]/jre/lib/security/, as the starting point for any truststore you create for the cluster.
The following instructions assume you have the version of keytool compatible with the version of JDK your cluster is running on.
Option A - Explicitly import the Cloudera Support server's certificate into your truststore
Use the following command to obtain the current public key/certificate information from the Cloudera Support server.
$ openssl s_client -connect cops.cloudera.com:443 | openssl x509 -text -out /path/to/cloudera-cert.pem
Import this certificate into your Cloudera Manager truststore. Substitute the paths in the following command with the paths to your truststore and the Support server's certificate.
$ keytool -import -keystore /path/to/cm/truststore.jks -file /path/to/cloudera-cert.pemOnce the truststore is ready, make sure Cloudera Manager is configured to point to this file. For instructions, see Configuring Cloudera Manager Truststore Properties.
Option B - Use the default cacerts truststore to create the Cloudera Manager truststore
- Copy the contents of cacerts to jssecacerts, and use jssecacerts as the Cloudera Manager
truststore. You can modify jssecacerts to include any additional private CA certificates as needed.
- As the first step to setting up the Cloudera Manager truststore, copy the contents of cacerts to your blank truststore.jks file.
$ keytool -storepasswd -keystore /path/to/cm/truststore.jks Enter keystore password: changeit New keystore password: [new-password] Re-enter new keystore password: [new-password]Once the truststore is ready, use the instructions in the following section to make sure Cloudera Manager is configured to point to the right truststore file.
Configuring Cloudera Manager Truststore Properties
Once you have set up the Cloudera Manager truststore using either Option A or B, make sure Cloudera Manager is configured to use this truststore. To determine trust, Cloudera Manager uses the JVM arguments, -Djavax.net.ssl.trustStore and -Djavax.net.ssl.trustStore.password. Configure these arguments in Cloudera Manager as follows:
- Log into the Cloudera Manager Admin Console.
- Select .
- Click the Security category.
- Configure the following TLS/SSL settings:
Setting Description Cloudera Manager TLS/SSL Certificate Trust Store File Specify the complete filesystem path to the truststore located on the Cloudera Manager Server host in .jks format. Cloudera Manager TLS/SSL Certificate Trust Store Password Specify the password for the truststore file. This password is not required to access the trust store. This field can be left blank.
- Click Save Changes to save the settings.