Amazon EMR
Management Guide

Troubleshoot Spot Instances

The following topics address issues that may arise when you use Spot Instances. For additional information on how to debug cluster issues, see Troubleshoot a Cluster.

Why haven’t I received my Spot Instances?

Spot Instances are provisioned based on availability and bid price. If you haven’t received the Spot Instances you requested, that means either your bid price is lower than the current Spot Price, or there is not enough supply at your bid price to fulfill your request.

Master and core instance groups are not fulfilled until all of the requested instances can be provisioned. Task nodes are fulfilled as they become available.

One way to address unfulfilled Spot Instance requests is to terminate the cluster and launch a new one, specifying a higher bid price. Reviewing the price history on Spot Instances tells you which bids have been successful in the recent past and can help you determine which bid is the best balance of cost savings and likelihood of being fulfilled. To review the Spot Instance price history, go to Spot Instances on the Amazon EC2 Pricing page.

Another option is to change the type of instance you request. For example, if you requested four Extra Large instances and the request has not been filled after a period of time, you might consider relaunching the cluster and placing a request for four Large instances instead. Because the base rate is different for each instance type, you would want to adjust your bid price accordingly. For example, if you bid 80% of the on-demand rate for an Extra Large instance you might choose to adjust your bid price on the new Spot Instance request to reflect 80% of the on-demand rate for a Large instance.

The final variable in the fulfillment of a Spot Instance request is whether there is unused capacity in your region. You can try launching the Spot Instance request in a different region. Before selecting this option, however, consider the implications of data transfer across regions. For example, if the Amazon Simple Storage (Amazon S3) bucket housing your data is in region us-east-1 and you launch a cluster as Spot Instances in us-west-1, the additional cross-region data transfer costs may outweigh any cost savings from using Spot Instances.

Why did my Spot Instances terminate?

By design, Spot Instances are terminated by Amazon EC2 when the Spot Instance price rises above your bid price.

If your bid price is equal to or lower than the Spot Instance price, the instances might have terminated normally at the end of the cluster, or they might have terminated because of an error. For more information about how to debug cluster errors, go to Troubleshoot a Cluster.

How do I check the price history on Spot Instances?

To review the Spot Instance price history, go to Spot Instances on the Amazon EC2 Pricing page. This pricing information is updated at regular intervals.