Spark supports the following means of encrypting Spark data at rest, and data in transit.
Enabling Spark Encryption Using Cloudera Manager
Minimum Required Role: Configurator (also provided by Cluster Administrator, Full Administrator)
- Open the Cloudera Manager Admin Console and go to the Spark service.
- Click the Configuration tab.
- (Prerequisite) Search for the Spark Authentication property and make sure it has been enabled. If this property is not set, the following settings to enable encryption will not work.
- Search for the Enable Network Encryption property. Use the checkbox to enable encrypted communication between Spark processes belonging to the same application.
- Search for the Enable I/O Encryption property. Use the checkbox to enabled encryption for temporary shuffle and cache files stored by Spark on local disks.
- Enter a Reason for change, and then click Save Changes to commit the changes.
- Redeploy client configuration.
- Restart stale services (if indicated by Cloudera Manager).