Cloudera Director 2.5.0
Unlock the Full Potential of Hadoop in the Cloud
Cloudera Director enables an enterprise-grade, self-service experience for deploying and managing CDH and Cloudera Enterprise in cloud environments.
Take advantage of CDH in the cloud.
Cloudera Director provides a single administration interface for central IT to deliver agility and for end-users to self-service provision and elastically scale clusters, all while ensuring auditability.
Cloudera Director runs as a web application. You can download and install the Cloudera Director server and client by selecting Standard Installation in the dropdown above. If you are new to Cloudera Director, you can get started quickly by selecting AWS Quick Start and following the wizard.
- System Requirements
- What's New
- Documentation
System Requirements
- Cloud Providers
- Cloudera Director Service Provider Interface (SPI)
- Supported Software and Distributions
- Resource Requirements
- Supported Cloudera Manager and CDH Versions
- Networking and Security Requirements
- Supported Browsers
Cloud Providers
Cloudera Director has native support for Amazon Web Services (AWS), Google Cloud Platform, and Microsoft Azure.
Each Cloudera Director release embeds the current plug-in for supported cloud providers, but a newer plug-in may have been posted on the Cloudera GitHub site subsequent to the Cloudera Director release. To check for the latest version, click the appropriate link:
- AWS cloud provider plug-in
- Google Cloud Platform cloud provider plug-in
- Microsoft Azure cloud provider plug-in
Cloudera Director Service Provider Interface (SPI)
The Cloudera Director SPI defines an open source Java interface that plug-ins implement to add support for additional cloud providers to Cloudera Director. For more information, see the README.md file in the SPI Cloudera Director GitHub repository.
Supported Software and Distributions
The table below lists software requirements, recommendations, and supported versions for resources used with Cloudera Director.
| Cloudera Director | Cloudera Manager and CDH | |
|---|---|---|
| Operating Systems (64-bit only) | RHEL and CentOS 6.5, 6.7, 6.8, 7.1, 7.2, and 7.3 Ubuntu 14.04 |
For AWS and Google Cloud Platform: RHEL and CentOS 6.5, 6.7, 6.8, 7.1, 7.2, and 7.3 For Microsoft Azure: RHEL and CentOS 6.7, 6.8, and 7.2 RHEL 7.2 is supported only for Cloudera Manager and CDH 5.7 and higher, not for lower versions of Cloudera Manager and CDH. To use Amazon EC2 D2 instances, you must run a minimum version of RHEL 6.7 or CentOS 6.7. Earlier versions of RHEL and CentOS do not support these instance types. |
| Oracle Java SE Development Kit (JDK) | Oracle JDK version 7 or 8 For download and installation information, see Java SE Downloads. |
Oracle JDK version 7 or 8 |
| Default Database | Embedded H2 database (not recommended for production use) | Embedded PostgreSQL Database (not recommended for production use) |
| Supported Databases | MySQL 5.5, 5.6, 5.7 MariaDB 5.5 |
MySQL 5.5, 5.6, 5.7 MariaDB 5.5 PostgreSQL 8.1, 8.3, 8.4, 9.1, 9.2, 9.3, 9.4, 9.5 |
Note: In production environments, you should use an external MySQL or MariaDB database for Cloudera Director. For information on using an external MySQL database in place of the H2 embedded database, see Using MySQL for Cloudera Director Server. For information on using an external MariaDB database in place of the H2 embedded database, see Using MariaDB for Cloudera Director Server. By default, Cloudera Director stores its environment and cluster data in the embedded H2 database located at /var/lib/cloudera-director-server/state.h2.db. Back up this file to avoid losing the data. Cloudera strongly recommends using MySQL or MariaDB for production deployments of Cloudera Director, instead of H2. Use of the H2 database in production environments can result in excessive space consumption for database files and slow database access. Unlike managed MySQL and MariaDB databases, H2 files are not backed up regularly, which puts your production deployment of Director at risk of data loss.
Note: The versions of PostgreSQL listed above are supported with Cloudera Manager and CDH 5.11. Setting up PostgreSQL via Amazon RDS for Cloudera Manager and CDH is not supported. For a table of PostgreSQL versions supported with earlier versions of Cloudera Manager and CDH, see the PostgreSQL section of CDH and Cloudera Manager Supported Databases in the Cloudera Enterprise release notes. For information on setting up external database servers and on creating databases on existing database servers, see Using an External Database for Cloudera Manager and CDH.
Note: To run Kafka and Sentry on the same cluster, you must use Kafka 2.1 with Cloudera Manager and CDH 5.9 or 5.10.
Note: For the latest information on operating system versions supported on Microsoft Azure, refer to the Cloudera Reference Architecture for Microsoft Azure Deployments.
Resource Requirements
The table below lists requirements for resources used with Cloudera Director.
| Cloudera Director | Cloudera Manager and CDH | |
|---|---|---|
| CPU | 2 | 4 |
| RAM | 3.75 GB | 64 GB |
| Disk | 8 GB | 500 GB |
| Recommended AWS instance | c3.large or c4.large | Cloudera Manager: m4.xlarge or m4.4xlarge |
| Recommended Google Cloud Platform instance | n1-standard-2 | n1-highmem-4 or n1-highmem-8 |
| Recommended Microsoft Azure instance | Standard_D3 or larger | The following Azure instance types are supported:
|
Note: For the latest information on instance types supported on Microsoft Azure, refer to the Cloudera Reference Architecture for Microsoft Azure Deployments.
Note: The recommended instance for Cloudera Manager depends on the workload. Some instance types may not be available in every region. Cloudera Director does not dynamically validate instance type by region. Contact your Cloudera account representative for more information.
Supported Cloudera Manager and CDH Versions
Cloudera Director 2.5 can install any version of Cloudera Manager 5 with any CDH 5 parcels. Use of CDH packages is not supported.
If you are using Cloudera Director 2.5 to deploy Cloudera Manager and CDH, the latest released version of Cloudera Manager 5.12 and CDH 5.12 is installed by default. To use any other version of Cloudera Manager or CDH, follow the instructions for installing non-default versions of Cloudera Manager and CDH in the Getting Started section for your cloud provider:
- For AWS, see Deploying Cloudera Manager and CDH on AWS.
- For Google Cloud Platform, see Deploying Cloudera Manager and CDH on Google Compute Engine.
- For Microsoft Azure, see Deploying Cloudera Manager and CDH on Microsoft Azure.
Networking and Security Requirements
Cloudera Director recommends the following inbound ports to be open:
- TCP ports 22: These ports allow SSH to Cloudera Director instance.
- All traffic across all ports within the security group: This rule allows connectivity with all the components within the Hadoop cluster. This rule avoids numerous individual ports to be opened in the security group.
| Type | Protocol | Port Range | Source |
|---|---|---|---|
| SSH (22) | TCP (6) | 22 | 0.0.0.0/0 |
| ALL Traffic | ALL | ALL | security_group_id See note paragraph below. |
Note: In AWS, the All traffic rule above requires the security group ID. If you create a security group from scratch, create the security group with the SSH rule and then go back and edit the security group to allow all traffic within the security group.
To connect to the AWS network, Cloudera recommends that you open only these ports and set up a SOCKS proxy. Unless your network has direct connection to AWS, you must set this up to access the Cloudera Director instance. This is done in a later step.
In a restricted network environment, you may want to enable minimal network traffic between instances and keep open ports to a minimum rather than enabling all network traffic between cluster instances. For information about minimal port requirements, see Ports Used by Cloudera Director.
Supported Browsers
Cloudera Director supports the following browsers:
- Mozilla Firefox 11 and higher
- Google Chrome
- Internet Explorer 9 and higher
- Safari 5 and higher
What's New
What's New in Cloudera Director 2.5.0
- Automatic repair: Cloudera Director can now automatically repair instances that get terminated unexpectedly for clusters that have auto-repair enabled.
- Significant improvement in resilience to unexpected instance terminations during bootstrap and grow operations. This improves support for AWS spot instances, as well as resilience of clusters using on-demand instances.
- Cloudera Director can now configure and create external accounts in Cloudera Manager that include cloud provider login credentials in order to manage the use of additional cloud provider functionality, including object stores, such as Amazon S3 in AWS.
- Cloudera Director can now recognize changes made directly from the cloud provider console to provisioned instances such as instance type and AMI.
- Cluster details page in the web UI now shows cluster history, displaying the most recent updates, including instances added, removed, or replaced in the cluster.
- Logging improvements, including additional information about service and role configuration errors returned by the Cloudera Manager API.
- A number of issues have been fixed. See Issues Fixed in Cloudera Director 2.5.0 for details.
Documentation
Want to Get Involved or Learn More?
Check out our other resources
Cloudera Community
Collaborate with your peers, industry experts, and Clouderans to make the most of your investment in Hadoop.
Cloudera University
Receive expert Hadoop training through Cloudera University, the industry's only truly dynamic Hadoop training curriculum that’s updated regularly to reflect the state of the art in big data.