This is the documentation for Cloudera 5.4.x. Documentation for other versions is available at Cloudera Documentation.

Managing Spark Using Cloudera Manager

Spark is available as two types of services: Spark and Spark (Standalone). In Cloudera Manager 5.1 and lower, the Spark service runs a Spark Standalone cluster, which has Master and Worker roles. In Cloudera Manager 5.2 the service that runs a Spark Standalone cluster has been renamed Spark (Standalone) and the Spark service runs Spark as a YARN application with only Gateway roles. Both services have a History Server role.

You can install Spark through the Cloudera Manager Installation wizard using parcels and have the Spark service added and started as part of the Installation wizard. See Installing Spark.

If you elect not to add the Spark service using the Installation wizard, you can use the Add Service wizard to create the service. The wizard automatically configures dependent services and the Spark service. See Adding a Service for instructions.

When you upgrade from Cloudera Manager 5.1 or lower to Cloudera 5.2 or higher, Cloudera Manager does not migrate an existing Spark service, which runs Spark Standalone, to a Spark on YARN service.

For information on Spark applications, see Spark Applications.