Configuring KMS Access Control Lists

Hadoop KMS supports a range of ACLs that control access to keys and key operations on a granular basis. ACLs can be used, for instance, to only grant users access to certain keys. Restricting HDFS superusers from access to key material is an important design requirement. This prevents a malicious superuser from having access to all the key material and all the encrypted data, and thus being able to decrypt everything.

There are two categories of KMS ACLs:

  1. KMS-wide: These ACLs specify the types of operations a user can perform. They are configured using the hadoop.kms.acl.<OPERATION> and hadoop.kms.blacklist.<OPERATION> parameters. The operations are as follows:
    • CREATE
    • DELETE
    • ROLLOVER
    • GET
    • GET_KEYS
    • GET_METADATA
    • SET_KEY_MATERIAL
    • GENERATE_EEK
    • DECRYPT_EEK
  2. Key-specific: These ACLs are set in a per-key basis. They are configured using the default.key.acl.<OPERATION>, whitelist.key.acl.<OPERATION>, and key.acl.<key_name>.<OPERATION> parameters. The operations and their programmatic equivalents are as follows:
    • READ - getKeyVersion, getKeyVersions, getMetadata, getKeysMetadata, getCurrentKey
    • MANAGEMENT - createKey, deleteKey, rolloverNewVersion
    • GENERATE_EEK - generateEncryptedKey, warmUpEncryptedKeys
    • DECRYPT_EEK - decryptEncryptedKey
    • ALL - All of the above

    The default.key.acl.<OPERATION> ACL applies to all keys for which an ACL has not been explicitly configured.

    If no ACL is configured for a specific key, and no default ACL is configured for the requested operation, access is denied.

The KMS supports both whitelist and blacklist ACLs. Blacklist entries override whitelist entries. A user or group accessing the KMS is first checked for inclusion in the ACL for the requested operation and then checked for exclusion in the blacklist for the operation before access is granted.

The group membership used by ACL entries relies on the configured group mapping mechanism for HDFS. By default, group membership is determined on the local Linux system running the KMS service. If you have configured HDFS to use LDAP for group mapping, the group membership for the ACL entries is determined using the configured LDAP settings. For more information about LDAP-based group membership, see Configuring LDAP Group Mappings.

The ACL syntax for both blacklist and whitelist entries is as follows:
  • Users only:
    user1,user2,userN
  • Groups only:
    nobody group1,group2,groupN
  • Users and Groups:
    user1,user2,userN group1,group2,groupN

Configuring KMS Access Control Lists Using Cloudera Manager

Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)

The KMS installation wizard includes an option to generate the recommended ACLs. To view or edit the ACLs:

  1. Go to the KMS service.
  2. Click Configuration.
  3. In the Search field, type acl to show the Key Management Server Advanced Configuration Snippet (Safety Valve) for kms-acls.xml (in the Key Management Server Default Group category).
  4. Add or edit the ACL properties according to your cluster configuration. See Recommended KMS Access Control List for example ACL entries.
  5. Click Save Changes.
  6. Return to the Home page by clicking the Cloudera Manager logo.
  7. Click the icon that is next to any stale services to invoke the cluster restart wizard.
  8. Click Restart Stale Services.
  9. Click Restart Now.
  10. Click Finish.

Configuring Java KeyStore KMS Access Control Lists Using the Command Line

KMS ACLs are defined in the /etc/hadoop-kms/conf/kms-acls.xml configuration file. This file is hot-reloaded when it changes. See Recommended KMS Access Control List for recommended ACL entries.

Recommended KMS Access Control List

Cloudera recommends the following ACL definition for secure production settings. Replace keyadmin and keyadmingroup with the user and group responsible for maintaining encryption keys.

<configuration>

<!--
  KMS ACLs control which users can perform various actions on the KMS,
  and which users and groups have access to which keys.

  This file has the following sections:
  * ACLs for KMS operations
  ** Access to specific KMS operations
  ** Blacklists for those specific operations
  * ACLs for keys
  ** Default ACLs for keys
  ** Whitelist ACLs for keys
  ** Key-specific ACLs
-->

<!--
  KMS ACLs that govern access to specific key operations. If access is not
  granted for an operation here, then the operation is forbidden, even if
  a key ACL allows it.

  The ACL value should be either a username or a username and group name
  separated by whitespace.

  A value of "*" (for the username or groupname) indicates that
  all users are granted access to that operation. Any operation for which
  there is no ACL or an empty (zero-length) ACL is treated as having an
  ACL with a value of "*". To disallow all users, add an ACL with a
  value of " ", a single space.

  Note: This convention applies only to the KMS-level ACLs beginning with
  'hadoop.kms.acl'.
-->

<property>
  <name>hadoop.kms.acl.CREATE</name>
  <value>keyadmin keyadmingroup</value>
  <description>
    ACL for create-key operations.
    If the user is not in the GET ACL, the key material is not returned
    as part of the response.
  </description>
</property>

<property>
  <name>hadoop.kms.acl.DELETE</name>
  <value>keyadmin keyadmingroup</value>
  <description>
    ACL for delete-key operations.
  </description>
</property>

<property>
  <name>hadoop.kms.acl.ROLLOVER</name>
  <value>keyadmin keyadmingroup</value>
  <description>
    ACL for rollover-key operations.
    If the user does is not in the GET ACL, the key material is not returned
    as part of the response.
  </description>
</property>

<property>
  <name>hadoop.kms.acl.GET</name>
  <value></value>
  <description>
    ACL for get-key-version and get-current-key operations.
  </description>
</property>

<property>
  <name>hadoop.kms.acl.GET_KEYS</name>
  <value>keyadmin keyadmingroup</value>
  <description>
    ACL for get-keys operations.
  </description>
</property>

<property>
  <name>hadoop.kms.acl.SET_KEY_MATERIAL</name>
  <value></value>
  <description>
    Complementary ACL for CREATE and ROLLOVER operations to allow the client
    to provide the key material when creating or rolling a key.
  </description>
</property>

<property>
  <name>hadoop.kms.acl.GENERATE_EEK</name>
  <value>hdfs supergroup</value>
  <description>
    ACL for generateEncryptedKey CryptoExtension operations.
  </description>
</property>

<!--
  KMS blacklists to prevent access to operations. These settings override the
  permissions granted by the KMS ACLs above.

  The blacklist value should be either a username or a username and group name
  separated by whitespace.

  A blank value indicates that no user is blacklisted from the operation. A
  value of "*" (for either the username or groupname) indicates that all users
  are blacklisted from the operation. Any operation for which there is no
  blacklist will be treated as having a blacklist with an empty value.
-->

<!--
  In this template the hdfs user is blacklisted for everything except
  GET_METADATA, GET_KEYS, and GENERATE_EEK. The GET and SET_KEY_MATERIAL
  operations are blacklisted for all users since Hadoop users should not
  need to perform those operations, and access to the key material should
  be as restricted as possible.
-->

<property>
  <name>hadoop.kms.blacklist.CREATE</name>
  <value>hdfs supergroup</value>
</property>

<property>
  <name>hadoop.kms.blacklist.DELETE</name>
  <value>hdfs supergroup</value>
</property>

<property>
  <name>hadoop.kms.blacklist.ROLLOVER</name>
  <value>hdfs supergroup</value>
</property>

<property>
  <name>hadoop.kms.blacklist.GET</name>
  <value>*</value>
</property>

<property>
  <name>hadoop.kms.blacklist.GET_KEYS</name>
  <value></value>
</property>

<property>
  <name>hadoop.kms.blacklist.SET_KEY_MATERIAL</name>
  <value>*</value>
</property>

<property>
  <name>hadoop.kms.blacklist.DECRYPT_EEK</name>
  <value>hdfs supergroup</value>
</property>

<property>
  <name>keytrustee.kms.acl.UNDELETE</name>
  <value></value>
  <description>
    ACL that grants access to the UNDELETE operation on all keys.
    Only used by Key Trustee KMS.
  </description>
</property>

<property>
  <name>keytrustee.kms.acl.PURGE</name>
  <value></value>
  <description>
    ACL that grants access to the PURGE operation on all keys.
    Only used by Key Trustee KMS.
  </description>
</property>

<!--
  Default key ACLs that govern access to key operations for key-operation pairs
  that do not have a specific key ACL already. Specific key ACLs will override
  the default key ACLs

  The ACL value should be either a username or a username and group name
  separated by whitespace.

  An empty value for an ACL indicates that no user is granted access to that
  operation. A value of "*" (for the username or groupname) indicates that
  all users are granted access to that operation. Any operation for which
  there is no ACL will be treated as having an ACL with an empty value.
-->

<property>
  <name>default.key.acl.MANAGEMENT</name>
  <value></value>
  <description>
    Default ACL that grants access to the MANAGEMENT operation on all keys.
  </description>
</property>

<property>
  <name>default.key.acl.GENERATE_EEK</name>
  <value></value>
  <description>
    Default ACL that grants access to the GENERATE_EEK operation on all keys.
  </description>
</property>

<property>
  <name>default.key.acl.DECRYPT_EEK</name>
  <value></value>
  <description>
    Default ACL that grants access to the DECRYPT_EEK operation on all keys.
  </description>
</property>

<property>
  <name>default.key.acl.READ</name>
  <value></value>
  <description>
    Default ACL that grants access to the READ operation on all keys.
  </description>
</property>

<!--
  Whitelist key ACLs that grant access to specific key operations. Any
  permissions granted here will be added to whatever permissions are granted
  by the specific key ACL or the default key ACL. Note that these whitelist
  ACLs grant access to operations on specific keys. If the operations
  themselves are not allowed because of the KMS ACLs/blacklists, then the
  operation will not be permitted, regardless of the whitelist settings.

  The ACL value should be either a username or a username and group name
  separated by whitespace.

  An empty value for an ACL indicates that no user is granted access to that
  operation. A value of "*" (for the username or groupname) indicates that
  all users are granted access to that operation. Any operation for which
  there is no ACL will be treated as having an ACL with an empty value.
-->

<property>
  <name>whitelist.key.acl.MANAGEMENT</name>
  <value>keyadmin keyadmingroup</value>
  <description>
    Whitelist ACL for MANAGEMENT operations for all keys.
  </description>
</property>

<property>
  <name>whitelist.key.acl.READ</name>
  <value>hdfs supergroup</value>
  <description>
    Whitelist ACL for READ operations for all keys.
  </description>
</property>

<property>
  <name>whitelist.key.acl.GENERATE_EEK</name>
  <value>hdfs supergroup</value>
  <description>
    Whitelist ACL for GENERATE_EEK operations for all keys.
  </description>
</property>

<property>
  <name>whitelist.key.acl.DECRYPT_EEK</name>
  <value>keyadmin keyadmingroup</value>
  <description>
    Whitelist ACL for DECRYPT_EEK operations for all keys.
  </description>
</property>

<!--
  Key ACLs that grant access to specific key operations. Any permissions
  granted here are added to whatever permissions are granted by the whitelists.
  The key ACL name should be key.acl.<keyname>.<OPERATION>.

  The ACL value should be either a username or a username and group name
  separated by whitespace.

  An empty value for an ACL indicates that no user is granted access to that
  operation. A value of "*" (for the username or groupname) indicates that
  all users are granted access to that operation. Any key operation for which
  there is no ACL will default to the default ACL for the operation.

  Normally adding users or groups for a specific key and DECRYPT_EEK is
  sufficient to allow access to data protected with HDFS data at rest
  encryption.
-->

<!--
  The following ACLs are required for proper functioning of services.
  CM does not create keys or encryption zones, however our best practices
  recommend encryption zones on certain directories. Below we assume that
  the user has followed our recommended naming scheme and named the keys
  according to our best practices: "hive-key" for the hive service,
  "hbase-key" for the hbase service, etc. If the key names are different,
  none of this will work out of the box, and you will need to edit these
  ACLs to match your key names.
-->

<property>
  <name>key.acl.hive-key.DECRYPT_EEK</name>
  <value>hive hive</value>
  <description>
    Gives the hive user and the hive group access to the key named "hive-key".
    This allows the hive service to read and write files in /user/hive/.
    Also note that the impala user ought to be a member of the hive group
    in order to enjoy this same access.
  </description>
</property>

<property>
  <name>key.acl.hive-key.READ</name>
  <value>hive hive</value>
  <description>
    Required because hive compares key strengths when joining tables.
  </description>
</property>

<property>
  <name>key.acl.hbase-key.DECRYPT_EEK</name>
  <value>hbase hbase</value>
  <description>
    Gives the hbase user and hbase group access to the key named "hbase-key".
    This allows the hbase service to read and write files in /hbase.
  </description>
</property>

<property>
  <name>key.acl.solr-key.DECRYPT_EEK</name>
  <value>solr solr</value>
  <description>
    Gives the solr user and solr group access to the key named "solr-key".
    This allows the solr service to read and write files in /solr.
  </description>
</property>

<property>
  <name>key.acl.mapred-key.DECRYPT_EEK</name>
  <value>mapred,yarn hadoop</value>
  <description>
    Gives the mapred user and mapred group access to the key named "mapred-key".
    This allows mapreduce to read and write files in /user/history.
    This is required by YARN.
  </description>
</property>

<property>
  <name>key.acl.hue-key.DECRYPT_EEK</name>
  <value>oozie,hue oozie,hue</value>
  <description>
    Gives the appropriate users and groups access to the key named "hue-key".
    This allows hue and oozie to read and write files in /user/hue.
    Oozie is required here because it will attempt to access workflows in
    /user/hue/oozie/workspaces.
  </description>
</property>



<!-- This example is required if there are encryption zones on user’s home
     directories. -->
<!--
<property>
  <name>key.acl.username-key.DECRYPT_EEK</name>
  <value>username username,hive,hbase,solr,oozie,hue,yarn</value>
  <description>
    Designed to be placed on a key that protects the EZ /user/username,
    and assumes that the key name is also "username-key", this shows that
    a number of services may want to reach in to access data. Remove
    those are are not needed for your use-case.
  </description>
</property>
-->

</configuration>

Configuring Java KeyStore KMS Delegation Tokens Using the Command Line

Configure KMS delegation token secret manager using the following properties:
  <property>
    <name>hadoop.kms.authentication.delegation-token.update-interval.sec</name>
    <value>86400</value>
    <description>
      How often the master key is rotated, in seconds. Default value 1 day.
    </description>
  </property>

  <property>
    <name>hadoop.kms.authentication.delegation-token.max-lifetime.sec</name>
    <value>604800</value>
    <description>
      Maximum lifetime of a delegation token, in seconds. Default value 7 days.
    </description>
  </property>

  <property>
    <name>hadoop.kms.authentication.delegation-token.renew-interval.sec</name>
    <value>86400</value>
    <description>
      Renewal interval of a delegation token, in seconds. Default value 1 day.
    </description>
  </property>

  <property>
    <name>hadoop.kms.authentication.delegation-token.removal-scan-interval.sec</name>
    <value>3600</value>
    <description>
      Scan interval to remove expired delegation tokens.
    </description>
  </property>