Azure Limits, Quotas, and Constraints
By default Azure limits the number of CPU cores that can be allocated in each region to 10. You must contact Microsoft Azure support to increase this limit before deploying a cluster using Cloudera Director. For more information, refer to the Azure subscription and service limits, quotas, and constraints page.
Not all Azure VM types are available in all Azure regions. Please refer to the Microsoft Azure page Products available by region to confirm if a VM type is available in a particular region. Please refer to Cloudera Reference Architecture on Azure for the latest supported VM types.
Azure Resources Managed by Cloudera Director
- A storage account for each VM.
- A NIC for each VM.
- A public IP address for each VM, if public IP addresses are enabled.
Deploying Production Clusters
While the Cloudera Director web UI can be used for proof-of-concept deployments on Azure, you must use the published sample configuration files for production deployments (see Useful Links below). You can modify the sample configuration file to fit your specific deployment environment, remove services you don’t need, and customize the sample bootstrap script. Configurations related to logging and data storage for individual services must not be changed. Deploying a cluster using the Cloudera Director command line interface and configuration file based on the examples ensures a repeatable deployment with the proper settings for Azure.
Please refer to the Cloudera Reference Architecture for Microsoft Azure Deployments document for more details.
Updating the Azure Plugin Timeout Value
- Download the latest supported azure-plugin.conf file from the Cloudera Director scripts repository.
- Find the parameter azure-backend-operation-polling-timeout-second in the provider section.
- Change the value to the desired duration in seconds.
- The storage account created by the plugin is used for the VM OS drive and cluster data drive. If you have manually attached a drive from a different storage account not created by the plugin, it will not be deleted.
- The NIC created by the plugin is attached to the VM. We assume that only one NIC is used per VM. Do not manually attach NICs to the VM created by the plugin.
- Deleting the NIC also deletes the public IP attached to the NIC. This includes public IPs created by Cloudera Director as well as public IPs attached manually.
- Cloudera Enterprise Reference Architecture for Azure Deployments.
- Configuration files for running Cloudera Director on Microsoft Azure:
- azure.simple.conf: This is a simple Cloudera Director configuration that creates a Cloudera Manager node and a four-node cluster (one master and three workers).
- azure.reference.conf: This is a reference Cloudera Director configuration that creates an eight-node cluster (three masters and five workers) with high availability (HA) enabled.
- azure.kerberos.conf: This is the same Cloudera Director configuration as the azure.reference.conf configuration, but with Kerberos enabled.