Using Spot Instances
To help you manage your cloud resource costs, Cloudera supports AWS Spot instances. Spot instances are Amazon EC2 instances that you can bid on. They run just like On-Demand instances, except that they are not provisioned until the instance price falls below your bid price. Amazon will terminate the instances when the instance price exceeds your bid price. Spot instances allow you to add capacity to your clusters at a low price.
- Hive on Spark
- Hive on MapReduce
Planning for Spot Instances
It is normal for Spot instances on a cluster to disappear over time. However, Cloudera Manager does not see that these instances are terminated. If you use Cloudera Manager to restart a cluster that contains a Spot instance group, and the Spot instances have terminated, the restart fails. If you are modifying any group in the cluster that has lost Spot instances, do not select the Restart checkbox.
- Delete the entire group.
- Add more instances to the group.
- Delete unprovisioned instances from the group (only as part of adding more instances to the group).
- Retry (repair) existing instances.
- Change the bid price for spot instances.
- Delete all instances without adding more, due to the minimum instance count requirement.
The bid price for Spot instances is set in an instance template. This template is associated with a group. Although you can modify the group, you cannot change the bid price. Therefore, if you set the bid price too low for successful provisioning, you must delete the group where that price is set and create a new group with the higher bid price. You must also delete the current group and create a new one if you want to drop the bid price.
Specifying Spot Instances
To specify Spot instances, create a new instance template and use this template for your instance group. In the Advanced Options section of the
Create New Instance Template wizard, check the Use Spot Instances checkbox and enter a value in the Spot
Spot Blocks: Specifying a Duration for Spot Instances
The AWS Spot block feature enables you to specify a fixed duration ranging from one to six hours for Spot instances. Spot instances with a predefined duration use a fixed hourly price that remains in effect for the Spot instance while it runs. The price for instances with a Spot block will not be as low as that for ordinary Spot instances, but will be lower than that of on-demand instances, and Spot block instances are guaranteed to run for the specified duration, after which they are terminated.
To configure an instance template for Spot block instances, check the Use Spot Instances checkbox, enter a value in the Spot
bid field, and choose a value from one to six hours (in intervals of 60 minutes) in the Spot Block Duration dropdown.
See Specifying a Duration for Your Spot Instances in the AWS documentation for more information about Spot blocks.
Best Practices for Using Spot Instances
- Use a Spot instance worker group in conjunction with an On-Demand worker group. This ensures that the cluster can redo computational tasks run on Spot instances that could be terminated before the tasks are finished.
- Use Spot instances only in contexts where the loss of the instance can be tolerated, as in a worker group. Do not use Spot instances for master nodes or for data storage.
- Use a minimum count of 0 for Spot instance groups. If you use a number above 0, the cluster will likely enter a failed state. If the cluster fails, contact Cloudera support for help.
Additional best practices for using spot instances can be found in Spot Instance Interruptions in the AWS documentation.