Upgrading Cloudera Navigator Key Trustee Server 3.8 to 5.5 Using the ktupgrade Script

Cloudera provides a Python script (ktupgrade) to simplify upgrading Key Trustee Server 3.8 to 5.5. The script upgrades package-based Key Trustee Server 3.8 to package-based Key Trustee Server 5.5 and switches the web server from Apache to CherryPy. After the upgrade completes, you must manually migrate Key Trustee Server to use parcels and be managed by Cloudera Manager.

CAUTION:
Cloudera recommends that you do not create new encryption keys, encryption zones, or clients during an upgrade.

To upgrade from 3.x to 5.5 manually, you must first upgrade to 5.4, and then upgrade to 5.5:

Prerequisites

  • Before upgrading Key Trustee Server, upgrade Cloudera Manager and CDH. See Cloudera Upgrade Overview and Upgrading CDH and Managed Services Using Cloudera Manager. If you are upgrading Key Trustee Server to a version higher than 5.5.2, you can upgrade Cloudera Manager and CDH directly to the version you want before continuing; you do not need to upgrade Cloudera Manager and CDH to 5.5 and complete the Key Trustee Server upgrade before upgrading Cloudera Manager and CDH to a higher version. The Cloudera Manager version must be equal to or higher than the Key Trustee Server version. See Product Compatibility Matrix for Cloudera Navigator Encryption for more information.
  • If you are using HDFS Transparent Encryption with Key Trustee Server, upgrade Key Trustee KMS. See Upgrading Key Trustee KMS for instructions.
  • You must run the ktupgrade script as root.
  • The ktupgrade script uses yum to upgrade Key Trustee Server. If the Key Trustee Server host does not have Internet access, you must download the Key Trustee Server dependencies from a host with Internet access and copy them to the Key Trustee Server host:
    1. Create a temporary directory to store the packages:
      $ mkdir tmp-keytrustee
    2. Download the bigtop-utils package from the CDH repository:
      $ sudo wget -P tmp-keytrustee <url>

      Replace <url> with the URL corresponding to the Key Trustee Server version to which you are upgrading:

      URL for bigtop-utils Package
      Key Trustee Server Version URL
      5.5.2 http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.5.2/RPMS/noarch/bigtop-utils-0.7.0+cdh5.5.2+0-1.cdh5.5.2.p0.10.el6.noarch.rpm
      5.5.0 http://archive.cloudera.com/cdh5/redhat/6/x86_64/cdh/5.5.0/RPMS/noarch/bigtop-utils-0.7.0+cdh5.5.0+0-1.cdh5.5.0.p0.15.el6.noarch.rpm
    3. Download the python-paste and python-cherrypy packages:
      $ sudo yum install yum-downloadonly
      $ sudo yum install --downloadonly --downloaddir=tmp-keytrustee/ python-paste python-cherrypy
    4. Copy the packages to the Key Trustee Server host:
      $ sudo scp tmp-keytrustee/*.rpm <username>@kts01.example.com:/path/to/tmp-keytrustee

      Replace kts01.example.com with the hostname of the active Key Trustee Server, and /path/to/tmp-keytrustee with the path to a directory to which you have access.

Download the ktupgrade Script and Repository Tarball

  1. Download the ktupgrade script on the active Key Trustee Server host:
    $ sudo wget http://archive.gazzang.com/keytrustee/ktupgrade

    If the Key Trustee Server host does not have Internet access, run the command on an Internet-connected host, and then copy the file to the active Key Trustee Server host.

  2. Download the repository tarball for Key Trustee Server 5.5.0 or 5.5.2:
    1. Select Packages from the SELECT DOWNLOAD TYPE drop-down menu.
    2. Select your operating system from the SELECT AN OS drop-down menu.
    3. Click DOWNLOAD NOW.
    4. Copy the downloaded file to the active Key Trustee Server host. Make sure you put the repository tarball and ktupgrade script in the same directory.

Run the ktupgrade Script

Upgrade the Active Key Trustee Server Using the ktupgrade Script

  1. On the active Key Trustee Server host, change to the directory that contains the ktupgrade script and the repository tarball:
    # cd /path/to/tmp-keytrustee

    If the host does not have Internet access, make sure that the dependency files you downloaded in Prerequisites are in the same directory as the script and tarball.

  2. Make sure the script is executable:
    # chmod a+x ktupgrade
  3. Run the ktupgrade script as follows:
    # ./ktupgrade upgrade-active-kts key-trustee-server-5.5.2-el6.tar.gz

    Replace key-trustee-server-5.5.2-el6.tar.gz with the file name of the repository tarball you downloaded in Download the ktupgrade Script and Repository Tarball.

Downgrade Key Trustee Server Using the ktupgrade Script

If you experience any problems upgrading Key Trustee Server, you can use the script to downgrade to your previous version. Run the following command on the active Key Trustee Server:

# cd /path/to/tmp-keytrustee
# ./ktupgrade downgrade-active-kts

Migrate Key Trustee Server to Cloudera Manager

Minimum Required Role: Cluster Administrator (also provided by Full Administrator)

Before continuing, you must create an internal repository for the Key Trustee Server parcel. For instructions on creating internal repositories (including Cloudera Manager, CDH, and Cloudera Navigator encryption components), see Creating and Using a Parcel Repository for Cloudera Manager.

After creating the internal Key Trustee Server parcel repository, do the following:

  1. Create a new cluster in Cloudera Manager containing only the Key Trustee Server hosts. This enables multiple clusters to share the same Key Trustee Server and avoids restarting Key Trustee Server when restarting a cluster. See Adding and Deleting Clusters for instructions on how to create a new cluster in Cloudera Manager.
  2. Download, distribute, and activate the Key Trustee Server parcel, following the instructions in Managing Parcels. After you activate the Key Trustee Server parcel, Cloudera Manager prompts you to restart the cluster. Click the Close button to ignore this prompt. You do not need to restart the cluster after installing Key Trustee Server.
  3. Stop the active and passive Key Trustee Server web servers by running the following command on all Key Trustee Server hosts:
    $ sudo -u keytrustee service keytrusteed stop
  4. Stop the active Key Trustee Server database by running the following command on the active Key Trustee Server:
    $ sudo -u keytrustee service keytrustee-db stop
  5. Add the Key Trustee Server service to your cluster, following the instructions in Adding a Service. When customizing role assignments, assign the Active Key Trustee Server and Active Database roles to the active Key Trustee Server host, and the Passive Key Trustee Server and Passive Database roles to the passive Key Trustee Server host.
  6. Stop the passive Key Trustee Server database. Run the following command on the passive Key Trustee Server:
    $ sudo -u keytrustee service keytrustee-db stop
  7. Restart the Key Trustee Server service (Key Trustee Server service > Actions > Restart).
  8. (High Availability Key Trustee Servers Only) Enable synchronous replication. Run the following command on the active Key Trustee Server:
    $ sudo -u keytrustee ktadmin enable-synchronous-replication --pg-rootdir /var/lib/keytrustee/db

Validate Key Operations

Verify that the upgrade was successful by running the following command on all Key Trustee Servers. The output should be similar to the following. If high availability is enabled, the output should be identical on all Key Trustee Servers:
$ curl -k https://keytrustee.example.com:11371/?a=fingerprint
4096R/4EDC46882386C827E20DEEA2D850ACA33BEDB0D1

Replace keytrustee.example.com with the fully qualified domain name (FQDN) of each Key Trustee Server you are validating.

If you are using Key Trustee Server as the backing key store for HDFS Transparent Encryption, run the following commands to verify that Hadoop key operations are successful:

$ hadoop key create hadoop_test_key
$ hadoop key list
$ hadoop key delete hadoop_test_key

Updating Key Trustee Server Clients

After upgrading Key Trustee Server to 5.4 or higher, you must configure Key Trustee Server clients (namely Key Trustee KMS and Cloudera Navigator Encrypt) to communicate with Key Trustee Server over the new ports:

  • Key Trustee KMS

    Add the following entries to the Key Trustee KMS advanced configuration snippet (Key Trustee KMS service > Configuration > Advanced > Key Management Server Advanced Configuration Snippet (Safety Valve) for kms-site.xml):

    <property>
        <name>cloudera.trustee.keyprovider.hkpport</name>
        <value>hkp_port_number</value>
        <description>
          Indicates the HTTP port on which Key Trustee Server clients should request public keys.
          On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually port 80 (unencrypted).
          On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually port 11371 (SSL-encrypted).
        </description>
    </property>
    <property>
        <name>cloudera.trustee.keyprovider.ktsport</name>
        <value>kts_port_number</value>
        <description>
          Indicates the HTTPS port on which the client sends and receives Key Trustee Server protocol messages.
          On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually port 443 (SSL-encrypted).
          On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually port 11371 (SSL-encrypted).
        </description>
    </property>
    <property>
        <name>cloudera.trustee.keyprovider.hkpssl</name>
        <value>boolean</value>
        <description>
          Indicates whether the client should communicate with the HKP server over an SSL-encrypted (true) or unencrypted (false) channel.
          On Key Trustee Server 3.8 (Apache webserver-based) servers, this is usually false (unencrypted).
          On Key Trustee Server 5.4 and higher (CherryPy-based) servers, this is usually true (SSL-encrypted).
        </description>
    </property>
  • Cloudera Navigator Encrypt

    See Updating Key Trustee Server Ports for instructions on updating Cloudera Navigator Encrypt to use the new ports.

Validating Key Operations

Verify that the upgrade was successful by running the following command on all Key Trustee Servers. The output should be similar to the following. If high availability is enabled, the output should be identical on all Key Trustee Servers:
$ curl -k https://keytrustee.example.com:11371/?a=fingerprint
4096R/4EDC46882386C827E20DEEA2D850ACA33BEDB0D1

Replace keytrustee.example.com with the fully qualified domain name (FQDN) of each Key Trustee Server you are validating.

If you are using Key Trustee Server as the backing key store for HDFS Transparent Encryption, run the following commands to verify that Hadoop key operations are successful:

$ hadoop key create hadoop_test_key
$ hadoop key list
$ hadoop key delete hadoop_test_key

(Optional) Upgrade to a Higher Release

If you are upgrading Key Trustee Server to a version higher than 5.5.2, continue to Upgrading Cloudera Navigator Key Trustee Server 5.4.x or Higher.