Enhanced snapshots as standalone resources Snapshot lifecycle Snapshot modes System snapshots Snapshot statuses Asynchronous operations Billing and quotas

How enhanced snapshots work

This page explains how enhanced snapshots work in Amazon Managed Service for Apache Flink. Read it before you upgrade an application to enhanced snapshot mode.

Note

Enhanced snapshots aren't available in the AWS Top Secret Cloud Regions or the AWS European Sovereign Cloud at this time.

Enhanced snapshots as standalone resources

An enhanced snapshot is an Apache Flink savepoint that Managed Service for Apache Flink stores as a standalone AWS resource. Each enhanced snapshot has its own ARN, lifecycle, tags, and IAM permissions.

Your account owns the enhanced snapshot, not the application. The ARN uses the resource type snapshot. The ARN does not contain the application name.

The ARN has the following format:


arn:partition:kinesisanalytics:region:account-id:snapshot/snapshot-name

The following example shows the ARN of an enhanced snapshot:


arn:aws:kinesisanalytics:aa-example-1:111122223333:snapshot/MyApplication-snapshot-1

Each enhanced snapshot name is unique in your account and Region. A name has 1 to 100 characters. Valid characters are letters, digits, underscores (_), periods (.), and hyphens (-).

The following rules apply to name reservation:

A name stays reserved in every status, including FAILED, until you delete the enhanced snapshot.
A create or import request that uses a reserved name returns ResourceInUseException.
The prefix SYSTEMSNAPSHOT is reserved for system snapshots, in any combination of uppercase and lowercase letters. A create or import request with a name that starts with this prefix returns InvalidArgumentException.

The following rules apply when you reuse a name:

After a delete completes, you can reuse the name for a new enhanced snapshot. The new enhanced snapshot contains none of the state of the deleted snapshot.
A new enhanced snapshot with a reused name has the same ARN as the deleted enhanced snapshot. An IAM policy that names that ARN applies to the new enhanced snapshot.
The tags of the deleted enhanced snapshot do not carry over to the new enhanced snapshot.
For a reused name, ListSnapshotOperations returns only the operations of the new enhanced snapshot.
A DeleteSnapshot request must set SnapshotCreationTimestamp. Use the creation time of the enhanced snapshot that you want to delete. If the value does not match, DeleteSnapshot returns InvalidArgumentException.

You can restore an enhanced snapshot into a different application in the same account and Region. Use this capability for blue/green deployments, A/B tests, and recovery. In a blue/green deployment, you run a new application next to the current application and then move traffic to the new application. For more information, see Restore an application from an enhanced snapshot.

Enhanced snapshots require Apache Flink version 1.20 or later. For more information, see Prerequisites.

Each enhanced snapshot belongs to one Region and one partition. At this time, Managed Service for Apache Flink does not support enhanced snapshot operations across Regions or partitions.

Snapshot lifecycle

The lifecycle of an enhanced snapshot does not depend on the application that created it. The enhanced snapshot remains after you stop, update, or delete that application.

After you delete an application, its enhanced snapshots remain, including its system snapshots. To remove them, call DeleteSnapshot for each enhanced snapshot.

Managed Service for Apache Flink does not expire or delete enhanced snapshots on a schedule. To manage retention, call DeleteSnapshot. For more information, see Create and manage enhanced snapshots.

When you close your AWS account, Managed Service for Apache Flink deletes your enhanced snapshots and your applications.

Note

To restore or export an enhanced snapshot that a customer managed key encrypts, the key policy must have the ARN of the application that created the enhanced snapshot. This rule applies after you delete that application. For more information about key policies, see Key management in Amazon Managed Service for Apache Flink.

Snapshot modes

The SnapshotMode setting of an application controls which type of snapshot the application uses. An application uses exactly one mode at a time.

SnapshotMode has the following values:

APPLICATION_SNAPSHOTS_LEGACY: The application uses application snapshots (legacy).
ENHANCED_SNAPSHOTS: The application uses enhanced snapshots.

The default value is APPLICATION_SNAPSHOTS_LEGACY.

Managed Service for Apache Flink does not put new applications in enhanced snapshot mode automatically. To use enhanced snapshot mode, set SnapshotMode yourself. An application with no stored mode reports APPLICATION_SNAPSHOTS_LEGACY.

You set the mode in the following request members:

CreateApplication: ApplicationConfiguration.ApplicationSnapshotConfiguration.SnapshotMode. For more information, see Set up enhanced snapshots.
UpdateApplication: ApplicationConfigurationUpdate.ApplicationSnapshotConfigurationUpdate.SnapshotModeUpdate. For more information, see Upgrade an existing application to enhanced snapshots.

To see the mode, call DescribeApplication. The response shows the mode in ApplicationDetail.ApplicationConfigurationDescription.ApplicationSnapshotConfigurationDescription.SnapshotMode. The following example shows that part of the response:


{
    "ApplicationDetail": {
        "ApplicationConfigurationDescription": {
            "ApplicationSnapshotConfigurationDescription": {
                "SnapshotsEnabled": true,
                "SnapshotMode": "ENHANCED_SNAPSHOTS"
            }
        }
    }
}

SnapshotsEnabled keeps its existing meaning in enhanced snapshot mode. The setting turns system snapshots on or off. If SnapshotsEnabled is false, you can still call CreateSnapshot.

System snapshots

In enhanced snapshot mode, a system snapshot is an enhanced snapshot that Managed Service for Apache Flink creates for you. If SnapshotsEnabled is true, Managed Service for Apache Flink creates a system snapshot for the following operations:

You stop a RUNNING application with StopApplication. A stop with Force set to true does not create a system snapshot.
You update the application with UpdateApplication.
For a RUNNING application, you add or delete a VPC configuration or a CloudWatch logging option.

The kinesisanalytics:CreateSystemSnapshot permission applies only when both of the following are true:

The application is in enhanced snapshot mode.
SnapshotsEnabled is true.

In that case, the principal that calls the following API actions needs the kinesisanalytics:CreateSystemSnapshot permission on the application ARN:

StartApplication: In any application status.
StopApplication: Only when the application is RUNNING. A StopApplication request with Force set to true does not need the permission.
UpdateApplication: Only when the application is RUNNING. If the request includes SnapshotModeUpdate or SnapshotsEnabledUpdate, Managed Service for Apache Flink uses those new values to decide whether the permission applies.
AddApplicationVpcConfiguration: Only when the application is RUNNING.
DeleteApplicationVpcConfiguration: Only when the application is RUNNING.
AddApplicationCloudWatchLoggingOption: Only when the application is RUNNING.
DeleteApplicationCloudWatchLoggingOption: Only when the application is RUNNING.

CreateApplication and RollbackApplication do not check the kinesisanalytics:CreateSystemSnapshot permission. For an IAM policy that grants the kinesisanalytics:CreateSystemSnapshot permission, see IAM permissions.

Start, rollback, and delete operations do not create a system snapshot. StartApplication still checks the kinesisanalytics:CreateSystemSnapshot permission, as the previous list shows.

A system snapshot name has the following format. The timestamp is the creation time in epoch milliseconds, with 13 digits.


SYSTEMSNAPSHOT-OPERATION-application-name-timestamp-ms

The following example shows the name of a system snapshot from a stop operation:


SYSTEMSNAPSHOT-STOP-MyApplication-1767225600000

A system snapshot has the following field values:

SnapshotInitiator: SYSTEM
SnapshotInitiatorOperation: SystemCreateSnapshot

Only the name shows which operation created the system snapshot.

To list your system snapshots, call ListSnapshots with SnapshotInitiator set to SYSTEM. Each call accepts at most one filter. For more information, see Create and manage enhanced snapshots.

Managed Service for Apache Flink bills the storage of system snapshots at the same rate as other enhanced snapshots. For more information, see Billing for enhanced snapshots.

For each system snapshot that succeeds, ListSnapshotOperations returns one operation with the status SUCCESSFUL. A failed system snapshot has no snapshot operation.

The DescribeSnapshotOperation response for a system snapshot includes ApplicationOperationDescription. To get details of the application operation, call the DescribeApplicationOperation action with the ApplicationOperationId and ApplicationName values from ApplicationOperationDescription.

When StopApplication, UpdateApplication, or a request that adds or deletes a VPC configuration or a CloudWatch logging option creates a system snapshot, the system snapshot inherits the tags of the application. Managed Service for Apache Flink copies the tags before the system snapshot appears in ListSnapshots. If the application has no tags, the system snapshot has no tags. To add more tags to a system snapshot, call TagResource with the ARN of the system snapshot.

Snapshot statuses

The SnapshotStatus field shows the status of an enhanced snapshot. The following table describes each status.

Status	Meaning
`CREATING`	Managed Service for Apache Flink is creating or importing the enhanced snapshot. You cannot restore or export the enhanced snapshot yet.
`READY`	The enhanced snapshot is complete. You can restore, export, or delete it.
`DELETING`	Managed Service for Apache Flink is deleting the enhanced snapshot.
`FAILED`	The create or import failed. The enhanced snapshot keeps its name until you delete it.

An enhanced snapshot moves between statuses as follows:

Create or import – The enhanced snapshot starts in CREATING. It then moves to READY or FAILED.
Delete – A delete moves a READY or FAILED enhanced snapshot to DELETING. Managed Service for Apache Flink then removes the enhanced snapshot.
Export – An export does not change the status of the enhanced snapshot. Instead, ExportSnapshot creates a snapshot operation that tracks the export. To follow the export, check the status of that snapshot operation. A failed export leaves the enhanced snapshot in READY. For more information, see Monitor enhanced snapshot operations.

No DELETED status exists. After a delete completes, DescribeSnapshot returns ResourceNotFoundException.

The snapshot operation for a DeleteSnapshot request never has the status FAILED. For the rules that block a delete request, see Delete an enhanced snapshot.

Asynchronous operations

The following actions run asynchronously: CreateSnapshot, DeleteSnapshot, ImportSnapshot, and ExportSnapshot. Each action returns a response after Managed Service for Apache Flink accepts the request. Managed Service for Apache Flink then completes the work in the background.

Each response includes a SnapshotOperationId and a SnapshotOperationStatus. The following table shows the members that each response returns.

Action	Response members
`CreateSnapshot`	`SnapshotArn`, `SnapshotStatus`, `SnapshotOperationId`, `SnapshotOperationStatus`
`DeleteSnapshot`	`SnapshotArn`, `SnapshotStatus`, `SnapshotOperationId`, `SnapshotOperationStatus`
`ImportSnapshot`	`SnapshotArn`, `SnapshotStatus`, `SnapshotOperationId`, `SnapshotOperationStatus`
`ExportSnapshot`	`SnapshotOperationId`, `SnapshotOperationStatus`

The following example shows a CreateSnapshot response:


{
    "SnapshotArn": "arn:aws:kinesisanalytics:aa-example-1:111122223333:snapshot/MyApplication-snapshot-1",
    "SnapshotStatus": "CREATING",
    "SnapshotOperationId": "a1B2c3D4e5F6",
    "SnapshotOperationStatus": "IN_PROGRESS"
}

An asynchronous action returns an exception only if Managed Service for Apache Flink rejects the request. If the work fails later, the action does not return an exception. Instead, Managed Service for Apache Flink reports the failure on the snapshot operation. For some actions, Managed Service for Apache Flink also changes the enhanced snapshot status to FAILED.

The following table shows where a failure appears for each action.

Action	Enhanced snapshot status after a failure	Snapshot operation after a failure
`CreateSnapshot`	Changes to `FAILED`	`FAILED`, with `SnapshotOperationFailureDetails`
`ImportSnapshot`	Changes to `FAILED`	`FAILED`, with `SnapshotOperationFailureDetails`
`ExportSnapshot`	Stays `READY`	`FAILED`, with `SnapshotOperationFailureDetails`

A FAILED enhanced snapshot keeps its name until you delete it.

To check the result of an asynchronous action, call DescribeSnapshotOperation with the snapshot name and the SnapshotOperationId. For more information, see Monitor enhanced snapshot operations.

An import or an export runs until it completes.

Billing and quotas

This section describes how Managed Service for Apache Flink bills enhanced snapshots. The section also lists the limits on enhanced snapshot operations that run at the same time.

Limits on operations in progress

Managed Service for Apache Flink limits the number of operations that run at the same time. The limits apply to one application, one enhanced snapshot, or one account in a Region. The following table lists these limits.

Operation	Limit	Exception for the next request
`CreateSnapshot`	One enhanced snapshot in `CREATING` for each application	`ResourceInUseException`
`ExportSnapshot`	One export in progress for each enhanced snapshot, for all destinations combined	`LimitExceededException`
`ImportSnapshot` and `ExportSnapshot`	10 imports and exports combined in progress for each account in each Region	`LimitExceededException`

You can run only one export at a time for each enhanced snapshot, even to different destinations.

If you get one of these exceptions, wait for the operation in progress to finish. Then retry the request.

Billing for enhanced snapshots

Managed Service for Apache Flink bills the storage of enhanced snapshots at the same rate as application snapshots (legacy) of $0.023 per GB-month under durable application backups.

Enhanced snapshot storage appears on your bill under the usage type region-DurableApplicationBackups. For more information about current prices, see Per second billing in Managed Service for Apache Flink.

Managed Service for Apache Flink bills the storage of the following enhanced snapshots:

Enhanced snapshots that you create with CreateSnapshot
Enhanced snapshots that you import with ImportSnapshot
System snapshots. For the operations that create a system snapshot, see System snapshots.

Billing for an enhanced snapshot starts when its status changes to READY. Billing stops when its status changes from READY to DELETING. Managed Service for Apache Flink does not bill an enhanced snapshot in the CREATING or FAILED status.

Billing for enhanced snapshots differs from billing for application snapshots (legacy) in the following ways:

Imported snapshots – You pay for the storage of each enhanced snapshot that you import.
Transfers – Imports and exports have no transfer charge from Managed Service for Apache Flink. Amazon S3 charges apply to the bucket that an import reads or an export writes.

Note

You pay for the storage of each enhanced snapshot until you delete it. This rule applies to system snapshots and after you delete the application.

For more information about deleting enhanced snapshots, see Delete an enhanced snapshot.

For more information about current prices, see Amazon Managed Service for Apache Flink Pricing.

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Manage application backups using snapshots

Set up enhanced snapshots