Altus Environment

An Altus environment specifies access for Altus to your cloud provider account and identifies the resources in your cloud provider account that Altus clusters and jobs can access. The Altus environment assigned to an Altus user determines the resources available to the clusters and jobs that the user creates.

For example, you assign a user an Altus environment with resources in your AWS account. When the Altus user creates a cluster or submits a job, Altus creates the cluster and runs the job in the VPC in your AWS account defined in the Altus environment assigned to the user.

Likewise, if you assign a user an Altus environment that identifies Azure resources, Altus creates the cluster and runs the job in your Azure subscription using the resources defined in the environment assigned to the user.

If users must access different cloud resources, create an Altus environment for each set of resources to have better control over user access to the resources.

Environment Options

An Altus environment includes options that you can configure for the clusters and jobs that use the resources specified in the environment.

Enable Workload XM

Available for AWS and Azure environments

Enables Workload Analytics for jobs that run on clusters created in the Altus environment. Logs, metrics, and configuration details from finished jobs are sent to the Telemetry Publisher in Cloudera Manager. The Telemetry Publisher transmits the metrics to Workload XM, which uses the information to perform health checks and create baselines for jobs.

For more information about the Workload Analytics tool, see Workload Analytics

Enable Log Archive

Available for AWS and Azure environments

Enables Altus to archive logs in cloud object storage.

For jobs that run on clusters in AWS, Altus writes the cluster logs to S3 at five minute intervals. It copies the job logs to S3 after a job completes. If you enable this option, you must specify the name of the S3 bucket to use for archiving cluster and job logs. Altus creates the S3 bucket in the region selected for this environment. When Altus creates the EC2 instance policy for the Altus environment, it includes S3 policies to allow write access to the S3 bucket that you specify.

For jobs that run on clusters in Azure, Altus copies the job logs to ADLS after a job completes, based on the log archive folder location you set. If you enable this option, you must specify the name of the directory in ADLS Gen1 or Gen2 to use for archiving job logs.

If you do not enable the log archive option, the Altus cluster and job logs will not be archived. When the cluster is terminated, the logs will not be available. Without the logs, Cloudera support might not be able to help you debug a job failure.

Enable Secure Clusters

Available for AWS and Azure environments

Enables wire and at-rest encryption and authentication in Altus clusters.

When you enable secure clusters, Altus secures the following components of the cluster:
  • Communication between nodes in the cluster.

    Altus sets up Kerberos authentication to secure Hadoop RPC communication between the nodes in the Altus cluster. Altus uses a Kerberos principal named altus to run jobs in the cluster.

  • Communication between Cloudera Manager and the cluster.

    Altus sets up TLS to secure communication between the Cloudera Manager instance and the cluster.

  • Data stored in the cluster.

    For clusters on AWS, Altus encrypts data stored in all EBS volumes in the cluster except the root volume. Altus clusters are configured so that no sensitive data is written to the root volume. This enables Altus to exclude the root volume from data encryption without compromising the security of the cluster. You can provide the AWS KMS encryption key to encrypt the volumes. If you do not provide a key, Altus uses the default AWS managed key to encrypt the volumes. For more information about the system volume in the Altus cluster, see System Volume

    For clusters on Azure, Altus relies on the security provided by the role-based access control that you set up for the storage disks in your Altus clusters. Enabling secure clusters in Altus does not change the level of security configured for data stored in clusters on Azure.

  • Data Warehouse clusters

    Altus sets up an LDAP directory of Altus users and groups for authentication of users who access Data Warehouse clusters. When you create a Data Warehouse cluster, Altus generates a user ID and password for the cluster. Any user who accesses the cluster from a client tool must use the cluster credentials to be authenticated and allowed access to the cluster.

Other aspects of security for Altus clusters are not affected by the secure cluster option:
  • Communication between the Altus cluster and the cloud object storage is always secure. By default, Altus sets up TLS to secure communication between the cluster and object storage.

    For clusters on AWS, Altus uses TLS to secure communication between the cluster and Amazon S3.

    For clusters on Azure, Altus uses TLS to secure communication between the cluster and Azure Data Lake Store.

  • Altus does not encrypt data that you store in your cloud object storage. By default, Azure encrypts all data that you store in Azure Data Lake Store. To secure data that you store in AWS S3, use the default encryption for S3 buckets provided by AWS. For more information, see Amazon S3 Default Encryption for S3 Buckets.

Enable Public IPs

Available for AWS and Azure environments

Configures Altus to assign public IP addresses to the clusters created using this environment.

By default, Altus assigns private IP addresses when it creates a cluster. When you enable the Public IPs option, Altus assigns public IP addresses in addition to the private IP addresses.

With public addresses set up in the cluster, Altus can provide direct access to the clusters. For example, the Altus console displays the URL that you use to access the Cloudera Manager instance in a cluster through its public IP address. You can use the URL to immediately connect to Cloudera Manager using the credentials that Altus creates for you.

If you do not enable the Public IPs option, Altus assigns only private IP addresses to the cluster. The clusters will not be publicly accessible from external networks. Altus can display the URL for Cloudera Manager on the Altus console. However, Altus cannot access your private subnet, so you cannot connect to Cloudera Manager using the URL. With the Public IPs option not enabled, you also cannot use other Altus connectivity features, such as the ability to use Altus credentials to access an Altus Data Warehouse cluster through JDBC or to use the Altus socks-proxy command to set up a SOCKS proxy connection to Cloudera Manager. To access Cloudera Manager or use the connectivity features, you must configure your network to allow access to your private subnets, such as by setting up a private internet connection or by setting up a bastion host in your network.

Enable S3Guard Consistency

Available for AWS environments

Enables S3Guard for the S3 buckets used for data processed in clusters that use this environment.

S3Guard ensures a consistent view of data stored in Amazon S3 and makes data written to S3 immediately available for processing. Without S3Guard, multi-step workloads can fail if data from a previous step is not available in the next step.

S3Guard uses an Amazon DynamoDB database to store metadata. Amazon charges an hourly rate for this service. See Amazon DynamoDB Pricing.