Continue rolling back an update - AWS CloudFormation

Continue rolling back an update

A stack goes into the UPDATE_ROLLBACK_FAILED state when AWS CloudFormation can't roll back all changes during an update. For example, you might have a stack that begins to roll back to an old database instance that was deleted outside of CloudFormation. Because CloudFormation doesn't know that the database was deleted, it assumes that the database instance still exists and attempts to roll back to it, causing the update rollback to fail.

When a stack is in the UPDATE_ROLLBACK_FAILED state, you can continue to roll it back to a working state (UPDATE_ROLLBACK_COMPLETE). You can't update a stack that is in the UPDATE_ROLLBACK_FAILED state. However, if you can continue to roll it back, you can return the stack to its original settings and then try to update it again.

In most cases, you must fix the error that causes the update rollback to fail before you can continue to roll back your stack. For more information, see . In other cases, you can continue to roll back the update without any changes, for example when a stack operation times out.

Note

If you use nested stacks, rolling back the parent stack will attempt to roll back all the child stacks as well.

To continue rolling back an update (console)
  1. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

  2. Select the stack that you want to update, choose Stack actions, and then choose Continue update rollback.

    If none of the solutions in the Troubleshooting errors worked, you can use the advanced option to skip the resources that CloudFormation can't successfully roll back. You must look up and type the logical IDs of the resources that you want to skip. Specify only resources that went into the UPDATE_FAILED state during the UpdateRollback and not during the forward update.

    Warning

    CloudFormation sets the status of the specified resources to UPDATE_COMPLETE and continues to roll back the stack. After the rollback is complete, the state of the skipped resources will be inconsistent with the state of the resources in the stack template. Before performing another stack update, you must update the stack or resources to be consistent with each other. If you don't, subsequent stack updates might fail, and the stack will become unrecoverable.

    Specify the minimum number of resources required to successfully roll back your stack. For example, a failed resource update might cause dependent resources to fail. In this case, it might not be necessary to skip the dependent resources.

    To skip resources that are part of nested stacks, use the following format: NestedStackName.ResourceLogicalID. If you want to specify the logical ID of a stack resource (Type: AWS::CloudFormation::Stack) in the ResourcesToSkip list, then its corresponding embedded stack must be in one of the following states: DELETE_IN_PROGRESS, DELETE_COMPLETE, or DELETE_FAILED.

To continue rolling back an update (AWS CLI)

Using ResourcesToSkip to recover a nested stacks hierarchy

The following diagram shows a nested stacks hierarchy that is in the UPDATE_ROLLBACK_FAILED state. In this example, the WebInfra root stack has two nested stacks: WebInfra-Compute and WebInfra-Storage, which in turn have one or more nested stacks.


        A diagram showing a three-level nested stack hierarchy.
Note

The stack names in this example are truncated for simplicity. Child stack names are typically generated by CloudFormation and contain unique random strings, so actual names might not be user-friendly.

To successfully get the root stack into an operable state using continue-update-rollback, you must use the resources-to-skip parameter to skip resources that failed to rollback. In this example, resources-to-skip would include the following items:

  1. myCustom

  2. WebInfra-Compute-Asg.myAsg

  3. WebInfra-Compute-LB.myLoadBalancer

  4. WebInfra-Storage.DB

The following example is the full AWS CLI command:

$ aws cloudformation continue-update-rollback --stack-name WebInfra \ --resources-to-skip myCustom WebInfra-Compute-Asg.myAsg WebInfra-Compute-LB.myLoadBalancer WebInfra-Storage.DB

We specified resources from nested stacks by using the NestedStackName.ResourceLogicalID format, but for the resources of the root stack, such as myCustom, we specified only the logical ID.

Finding the stack name of a nested stack

You can find a child stack's name in its stack ID or Amazon Resource Name (ARN). In the following example, the stack name is WebInfra-Storage-Z2VKC706XKXT:

arn:aws:cloudformation:us-east-1:123456789012:stack/WebInfra-Storage-Z2VKC706XKXT/ea9e7f90-54f7-11e6-a032-028f3d2330bd

Finding the logical ID of a nested stack

You can find a child stack's logical ID in the template definition of its parent. In the diagram, the LogicalId of the WebInfra-Storage-DB child stack is DB in its parent WebInfra-Storage.

In the CloudFormation console, you can also find the logical ID in the Logical ID column for the stack resource on the Resources tab or the Events tab.