Frequently Asked Questions about CDS 2 Powered by Apache Spark

This Frequently Asked Questions (FAQ) page covers general information about CDS 2 Powered by Apache Spark, coexistence with Spark 1, and other questions that are relevant for early adopters of the latest Spark 2 features.

Running Spark 1 and Spark 2 Side-by-Side

The Spark 2 service does not conflict with Spark 1 if it is installed. The history server uses a different port. Spark 2 shares the Spark 1 shuffle service if already available, or installs the shuffle service if not.

Why doesn't feature or library XYZ work?

A number of features, components, libraries, and integration points from Spark 1.6 are not supported with the CDS 2 Powered by Apache Spark. See Spark 2 Known Issues for details.