Azure Subscription Requirements

Verify that your Azure subscription has resources and services configured to meet Altus requirements and the Altus administrator has adequate permissions to configure the resources and services in Azure.

Azure Resources and Services

To create Altus clusters and run jobs in Azure, you must have an Azure subscription . You can use an existing Azure subscription or create an Azure subscription for Altus workloads. If you use an existing subscription, verify that it has the services and resources required by Altus to create clusters and run jobs.

Altus accesses the following resources in your Azure subscription:
Virtual Network (VNet)
The Azure subscription must have at least one subnet with five available private network addresses. If you create clusters with a larger number of nodes, the subscription requires additional subnets and network addresses.
DNS server
Cloudera Altus requires that you use a DNS server provisioned by Cloudera Altus using the Altus ARM Template. For more information, see Step 1. Create the Azure Resources with the Altus ARM Template.
Network security group (NSG)

You must configure the network security group to allow outbound traffic to all destinations.

To connect to an Altus Data Engineering or Data Warehouse cluster through SSH, you must also configure the network security group inbound rules to allow SSH access from the IP address of your machine or a range of IP addresses used by your organization. For more information, see SSH Connection in Azure .

Azure Data Lake Store (ADLS)

Altus reads data from and writes data processed by your jobs to files in ADLS Gen1 or Gen2. You can use existing ADLS files or set up new files for the data that you process in Altus clusters.

You must configure access control lists (ACL) for the files and directories in your ADLS account to manage user access to data.

User-Assigned Service Identity

You can use an existing user-assigned service identity or create one to control access to files and directories in the Azure Data Lake Store account you use for Altus clusters.

Configure access control lists (ACLs) to grant the user-assigned service identity read access to the files and directories from which Altus reads data and write access to the files and directories to which Altus writes data. Cloudera recommends that you configure a default ACL on the folders in Azure Data Lake Store which grants read and write permission to the user-assigned service identity.

For information about setting up a user-assigned service identity, see the Microsoft documentation on how to create, list or delete a user-assigned identity using the Azure CLI. You can use the Azure CLI to create and manage a user-assigned service identity.

You can optionally add the user-assigned service identity to an Azure Active Directory (AAD) group and configure ACLs for the AAD group to allow access to the files and directories accessed by Altus clusters.

Azure Permissions

As an Altus administrator, you must be able to create and manage the resources in the Azure subscription where Altus users create clusters and run jobs. You must be able to perform all administrative tasks and have administrative rights to all resources.

Cloudera recommends that the Altus administrator has the role of Owner of the Azure subscription.

For more information about setting up an Azure subscription administrator, see Add an RBAC Owner admin for a subscription in Azure portal in the Azure documentation.

Azure Resources and Limits

Altus creates the following resources in your Azure subscription:
  • Managed disks
  • Network interfaces
  • Public IP addresses (optional)
  • Virtual machines
Depending on the number of clusters and the number of worker nodes per cluster that Altus creates in your Azure subscription, you might need to raise the limits for the following resources in your Azure subscription:
  • Standard DSv2, DSv3, and ESv3 Family Cores. Each virtual machine in a cluster consumes a number of cores, determined by the size of the virtual machine.
  • Total Regional Cores. The number of regional cores consumed by the cluster is determined by the size of the virtual machine.
  • Public IP Addresses. If you allow Altus to assign public IP addresses, each virtual machine in a cluster consumes one public IP address.

Supported Azure Regions

You can create Altus clusters in Azure regions where the ADLS service is available. The available regions depend on the ADLS version where cluster data is stored.

For clusters that use ADLS Gen 1, Altus supports the following Azure regions where ADLS Gen 1 is supported:
  • Australia Southeast (australiasoutheast)
  • Central US (centralus)
  • East US 2 (eastus2)
  • Japan East (japaneast)
  • North Europe (northeurope)
  • Southeast Asia (southeastasia)
  • West Europe (westeurope)
For Altus clusters that use ADLS Gen 2, Altus supports the following Azure regions where ADLS Gen 2 is supported:
  • Australia East (australiaeast)
  • Australia Southeast (australiasoutheast)
  • Brazil South (brazilsouth)
  • Canada Central (canadacentral)
  • Canada East (canadaeast)
  • Central India (centralindia)
  • Central US (centralus)
  • East Asia (eastasia)
  • East US 2 (eastus2)
  • Japan East (japaneast)
  • Japan West (japanwest)
  • Korea Central (koreacentral)
  • Korea South (koreasouth)
  • North Central US (northcentralus)
  • North Europe (northeurope)
  • South Central US (southcentralus)
  • Southeast Asia (southeastasia)
  • South India (southindia)
  • UK South (uksouth)
  • UK West (ukwest)
  • West Central US (westcentralus)
  • West Europe (westeurope)
  • West India (westindia)
  • West US (westus)
  • West US 2 (westus2)

For more information about the Microsoft Azure regions, see Products available by region on the Azure portal.