Conflict Detection and Sync
Versioned Data Sources
AWS AppSync currently supports versioning on DynamoDB data sources. Conflict Detection,
Conflict Resolution, and Sync operations require a Versioned
data source. When
you enable versioning on a data source, AWS AppSync will automatically:
-
Enhance items with object versioning metadata.
-
Record changes made to items with AWS AppSync mutations to a Delta table.
-
Maintain deleted items in the Base table with a “tombstone” for a configurable amount of time.
Versioned Data Source Configuration
When you enable versioning on a DynamoDB data source, you specify the following fields:
-
BaseTableTTL
-
The number of minutes to retain deleted items in the Base table with a “tombstone” - a metadata field indicating that the item has been deleted. You can set this value to 0 if you want items to be removed immediately when they are deleted. This field is required.
-
DeltaSyncTableName
-
The name of the table where changes made to items with AWS AppSync mutations are stored. This field is required.
-
DeltaSyncTableTTL
-
The number of minutes to retain items in the Delta table. This field is required.
Delta Sync Table
AWS AppSync currently supports Delta Sync Logging for mutations using
PutItem
, UpdateItem
, and DeleteItem
DynamoDB
operations.
When an AWS AppSync mutation changes an item in a versioned data source, a record of that change will be stored in a Delta table that is optimized for incremental updates. You can choose to use different Delta tables (e.g. one per type, one per domain area) for other versioned data sources or a single Delta table for your API. AWS AppSync recommends against using a single Delta table for multiple APIs to avoid the collision of primary keys.
The schema required for this table is as follows:
-
ds_pk
-
A string value that is used as the partition key. It is constructed by concatenating the Base data source name and the ISO 8601 format of the date the change occurred (e.g.
Comments:2019-01-01
).When the
customPartitionKey
flag from the VTL mapping template is set as the column name of the partition key (see Resolver Mapping Template Reference for DynamoDB in the AWS AppSync Developer Guide), the format ofds_pk
changes, and the string is constructed by appending it the value of the partition key in the new record in the Base table. For example, if the record in the Base table has a partition key value of1a
and a sort key value of2b
, the new value of the string will be:Comments:2019-01-01:1a
. -
ds_sk
-
A string value that is used as the sort key. It is constructed by concatenating the ISO 8601 format of the time the change occurred, the primary key of the item, and the version of the item. The combination of these fields guarantees uniqueness for every entry in the Delta table (e.g. for a time of
09:30:00
and an ID of1a
and version of2
, this would be09:30:00:1a:2
).When the
customPartitionKey
flag from the VTL mapping template is set to the column name of the partition key (see Resolver Mapping Template Reference for DynamoDB in the AWS AppSync Developer Guide), the format ofds_sk
changes, and the string is constructed by replacing the value of the combination key with the value of the sort key in the Base table. Using the previous example above, if the record in the Base table has a partition key value of1a
and a sort key value of2b
, the new value of the string will be:09:30:00:2b:3
. -
_ttl
-
A numeric value that stores the timestamp, in epoch seconds, when an item should be removed from the Delta table. This value is determined by adding the
DeltaSyncTableTTL
value configured on the data source to the moment when the change occurred. This field should be configured as the DynamoDB TTL Attribute.
The IAM role configured for use with the Base table must also contain permission to operate on the Delta table. In this example, the permissions
policy for a Base table called
Comments
and a Delta table
called ChangeLog
is displayed:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "dynamodb:DeleteItem", "dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:Query", "dynamodb:Scan", "dynamodb:UpdateItem" ], "Resource": [ "arn:aws:dynamodb:us-east-1:000000000000:table/Comments", "arn:aws:dynamodb:us-east-1:000000000000:table/Comments/*", "arn:aws:dynamodb:us-east-1:000000000000:table/ChangeLog", "arn:aws:dynamodb:us-east-1:000000000000:table/ChangeLog/*" ] } ] }
Versioned Data Source Metadata
AWS AppSync manages metadata fields on Versioned
data sources on your
behalf. Modifying these fields yourself may cause errors in your application or data
loss. These fields include:
-
_version
-
A monotonically increasing counter that is updated any time that a change occurs to an item.
-
_lastChangedAt
-
A numeric value that stores the timestamp, in epoch milliseconds, when an item was last modified.
-
_deleted
-
A Boolean “tombstone” value that indicates that an item has been deleted. This can be used by applications to evict deleted items from local data stores.
-
_ttl
-
A numeric value that stores the timestamp, in epoch seconds, when an item should be removed from the underlying data source.
-
ds_pk
-
A string value that is used as the partition key for Delta tables.
-
ds_sk
-
A string value that is used as the sort key for Delta tables.
gsi_ds_pk
-
A string value attribute that's generated to support a global secondary index as a partition key. It will be included only if both the
customPartitionKey
andpopulateIndexFields
flags are enabled in the VTL mapping template (see Resolver Mapping Template Reference for DynamoDB in the AWS AppSync Developer Guide). If enabled, the value will be constructed by concatenating the Base data source name and the ISO 8601 format of the date at which the change occurred (e.g. if the Base table is named Comments, this record will be set asComments:2019-01-01
). gsi_ds_sk
-
A string value attribute that's generated to support a global secondary index as a sort key. It will be included only if both the
customPartitionKey
andpopulateIndexFields
flags are enabled in the VTL mapping template (see Resolver Mapping Template Reference for DynamoDB in the AWS AppSync Developer Guide). If enabled, the value will be constructed by concatenating the ISO 8601 format of the time at which the change occurred, the partition key of the item in the Base table, the sort key of the item in the Base table, and the version of the item (e.g. for a time of09:30:00
, a partition key value of1a
, a sort key value of2b
, and version of3
, this would be09:30:00:1a#2b:3
).
These metadata fields will impact the overall size of items in the underlying data
source. AWS AppSync recommends reserving 500 bytes + Max
Primary Key Size of storage for versioned data source metadata when
designing your application. To use this metadata in client applications, include the
_version
, _lastChangedAt
, and _deleted
fields
on your GraphQL types and in the selection set for mutations.
Conflict Detection and Resolution
When concurrent writes happen with AWS AppSync, you can configure Conflict Detection and
Conflict Resolution strategies to handle updates appropriately. Conflict Detection
determines if the mutation is in conflict with the actual written item in the data source.
Conflict Detection is enabled by setting the value in the SyncConfig for the
conflictDetection
field to VERSION
.
Conflict Resolution is the action that is taken in the event that a conflict is detected. This is determined by setting the Conflict Handler field in the SyncConfig. There are three Conflict Resolution strategies:
-
OPTIMISTIC_CONCURRENCY
-
AUTOMERGE
-
LAMBDA
Each of these Conflict Resolution strategies are detailed in depth below.
Versions are automatically incremented by AppSync during write operations and should not be modified by clients or outside of a resolver configured with a version-enabled data source. Doing so will change the consistency behavior of the system and could result in data loss.
Optimistic Concurrency
Optimistic Concurrency is a conflict resolution strategy that AWS AppSync provides for versioned data sources. When the conflict resolver is set to Optimistic Concurrency, if an incoming mutation is detected to have a version that differs from the actual version of the object, the conflict handler will simply reject the incoming request. Inside the GraphQL response, the existing item on the server that has the latest version will be provided. The client is then expected to handle this conflict locally and retry the mutation with the updated version of the item.
Automerge
Automerge provides developers an easy way to configure a conflict resolution strategy without writing client-side logic to manually merge conflicts that were unable to be handled by other strategies. Automerge adheres to a strict rule set when merging data to resolve conflicts. The tenets of Automerge revolve around the underlying data type of the GraphQL field. They are as follows:
-
Conflict on a scalar field: GraphQL scalar or any field that is not a collection (i.e. List, Set, Map). Reject the incoming value for the scalar field and select the value existing in the server.
-
Conflict on a list: GraphQL type and database type are lists. Concatenate the incoming list with the existing list in the server. The list values in the incoming mutation will be appended to the end of the list in the server. Duplicate values will be retained.
-
Conflict on a set: GraphQL type is a list and database type is a Set. Apply a set union using incoming the set and the existing set in the server. This adheres to the properties of a Set, meaning no duplicate entries.
-
When an incoming mutation adds a new field to the item or is made against a field with the value of
null
, merge that on to the existing item. -
Conflict on a map: When the underlying data type in the database is a Map (i.e. key-value document), apply the above rules as it parses and processes each property of the Map.
Automerge is designed to automatically detect, merge, and retry requests with an updated version, absolving the client from needing to manually merge any conflicting data.
To show an example of how Automerge handles a Conflict on a Scalar type. We will use the following record as our starting point.
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "_version" : 4 }
Now an incoming mutation might be attempting to update the item but with an older version since the client has not synchronized with the server yet. That looks like this:
{ "id" : 1, "name" : "Nadia", "jersey" : 55, "_version" : 2 }
Notice the outdated version of 2 in the incoming request. During this flow, Automerge will merge the data by rejecting the ‘jersey’ field update to ‘55’ and keep the value at ‘5’ resulting in the following image of the item being saved in the server.
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "_version" : 5 # version is incremented every time automerge performs a merge that is stored on the server. }
Given the state of the item shown above at version 5, now suppose an incoming mutation that attempts to mutate the item with the following image:
{ "id" : 1, "name" : "Shaggy", "jersey" : 5, "interests" : ["breakfast", "lunch", "dinner"] # underlying data type is a Set "points": [24, 30, 27] # underlying data type is a List "_version" : 3 }
There are three points of interest in the incoming mutation. The name, a scalar, has been changed but two new fields “interests”, a Set, and “points”, a List, have been added. In this scenario, a conflict will be detected due to the version mismatch. Automerge adheres to its properties and rejects the name change due to it being a scalar and add on the non-conflicting fields. This results in the item that is saved in the server to appear as follows.
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "interests" : ["breakfast", "lunch", "dinner"] # underlying data type is a Set "points": [24, 30, 27] # underlying data type is a List "_version" : 6 }
With the updated image of the item with version 6, now suppose an incoming mutation (with another version mismatch) tries to transform the item to the following:
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "interests" : ["breakfast", "lunch", "brunch"] # underlying data type is a Set "points": [30, 35] # underlying data type is a List "_version" : 5 }
Here we observe that the incoming field for “interests” has one duplicate value that exists in the server and two new values. In this case, since the underlying data type is a Set, Automerge will combine the values existing in the server with the ones in the incoming request and strip out any duplicates. Similarly there is a conflict on the “points” field where there is one duplicate value and one new value. But since the underlying data type here is a List, Automerge will simply append all values in the incoming request to the end of the values already existing in the server. The resulting merged image stored on the server would appear as follows:
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "interests" : ["breakfast", "lunch", "dinner", "brunch"] # underlying data type is a Set "points": [24, 30, 27, 30, 35] # underlying data type is a List "_version" : 7 }
Now let’s assume the item stored in the server appears as follows, at version 8.
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "interests" : ["breakfast", "lunch", "dinner", "brunch"] # underlying data type is a Set "points": [24, 30, 27, 30, 35] # underlying data type is a List "stats": { "ppg": "35.4", "apg": "6.3" } "_version" : 8 }
But an incoming request tries to update the item with the following image, once again with a version mismatch:
{ "id" : 1, "name" : "Nadia", "stats": { "ppg": "25.7", "rpg": "6.9" } "_version" : 3 }
Now in this scenario, we can see that the fields that already exist in the server are missing (interests, points, jersey). In addition, the value for “ppg” within the map “stats” is being edited, a new value “rpg” is being added, and “apg” is omitted. Automerge preserve the fields that have been omitted (note: if fields are intended to be removed, then the request must be tried again with the matching version), and so they will not be lost. It will also apply the same rules to fields within maps and therefore the change to “ppg” will be rejected whereas “apg” is preserved and “rpg”, a new field”, is added on. The resulting item stored in the server will now appear as:
{ "id" : 1, "name" : "Nadia", "jersey" : 5, "interests" : ["breakfast", "lunch", "dinner", "brunch"] # underlying data type is a Set "points": [24, 30, 27, 30, 35] # underlying data type is a List "stats": { "ppg": "35.4", "apg": "6.3", "rpg": "6.9" } "_version" : 9 }
Lambda
Conflict Resolution options:
-
RESOLVE
: Replace the existing item with new item supplied in response payload. You can only retry the same operation on a single item at a time. Currently supported for DynamoDBPutItem
&UpdateItem
. -
REJECT
: Reject the mutation and returns an error with the existing item in the GraphQL response. Currently supported for DynamoDBPutItem
,UpdateItem
, &DeleteItem
. -
REMOVE
: Remove the existing item. Currently supported for DynamoDBDeleteItem
.
The Lambda Invocation Request
The AWS AppSync DynamoDB resolver invokes the Lambda function specified in the
LambdaConflictHandlerArn
. It uses the same service-role-arn
configured on the data source. The payload of the invocation has the following
structure:
{ "newItem": { ... }, "existingItem": {... }, "arguments": { ... }, "resolver": { ... }, "identity": { ... } }
The fields are defined as follows:
-
newItem
-
The preview item, if the mutation succeeded.
-
existingItem
-
The item currently resided in DynamoDB table.
-
arguments
-
The arguments from the GraphQL mutation.
-
resolver
-
Information about the AWS AppSync resolver.
-
identity
-
Information about the caller. This field is set to null, if access with API key.
Example payload:
{ "newItem": { "id": "1", "author": "Jeff", "title": "Foo Bar", "rating": 5, "comments": ["hello world"], }, "existingItem": { "id": "1", "author": "Foo", "rating": 5, "comments": ["old comment"] }, "arguments": { "id": "1", "author": "Jeff", "title": "Foo Bar", "comments": ["hello world"] }, "resolver": { "tableName": "post-table", "awsRegion": "us-west-2", "parentType": "Mutation", "field": "updatePost" }, "identity": { "accountId": "123456789012", "sourceIp": "x.x.x.x", "username": "AIDAAAAAAAAAAAAAAAAAA", "userArn": "arn:aws:iam::123456789012:user/appsync" } }
The Lambda Invocation Response
For PutItem
and UpdateItem
conflict resolution
RESOLVE
the mutation. The response must be in the following format.
{ "action": "RESOLVE", "item": { ... } }
The item
field represents an object that will be used to replace the
existing item in the underlying data source. The primary key and sync metadata will be
ignored if included in item
.
REJECT
the mutation. The response must be in the following format.
{ "action": "REJECT" }
For DeleteItem
conflict resolution
REMOVE
the item. The response must be in the following format.
{ "action": "REMOVE" }
REJECT
the mutation. The response must be in the following format.
{ "action": "REJECT" }
The example Lambda function below checks who makes the call and the resolver name. If
it is made by jeffTheAdmin
, REMOVE
the object for DeletePost
resolver or RESOLVE
the conflict with new item for Update/Put resolvers. If
not, the mutation is REJECT
.
exports.handler = async (event, context, callback) => { console.log("Event: "+ JSON.stringify(event)); // Business logic goes here. var response; if ( event.identity.user == "jeffTheAdmin" ) { let resolver = event.resolver.field; switch(resolver) { case "deletePost": response = { "action" : "REMOVE" } break; case "updatePost": case "createPost": response = { "action" : "RESOLVE", "item": event.newItem } break; default: response = { "action" : "REJECT" }; } } else { response = { "action" : "REJECT" }; } console.log("Response: "+ JSON.stringify(response)); return response; }
Errors
-
ConflictUnhandled
-
Conflict detection finds a version mismatch and the conflict handler rejects the mutation.
Example: Conflict resolution with an Optimistic Concurrency conflict handler. Or, Lambda conflict handler returned with
REJECT
. -
ConflictError
-
An internal error occurs when trying to resolve a conflict.
Example: Lambda conflict handler returned a malformed response. Or, cannot invoke Lambda conflict handler because the supplied resource
LambdaConflictHandlerArn
is not found. -
MaxConflicts
-
Max retry attempts were reached for conflict resolution.
Example: Too many concurrent requests on the same object. Before the conflict is resolved, the object is updated to a new version by another client.
-
BadRequest
-
Client tries to update metadata fields (
_version
,_ttl
,_lastChangedAt
,_deleted
).Example: Client tries to update _version of an object with an update mutation.
-
DeltaSyncWriteError
-
Failed to write delta sync record.
Example: Mutation succeeded, but an internal error occurred when trying to write to the delta sync table.
-
InternalFailure
-
An internal error occurred.
CloudWatch Logs
If an AWS AppSync API has enabled CloudWatch Logs with the logging settings set to Field-Level
Logs enabled
and log-level for the Field-Level Logs set to
ALL
, then AWS AppSync will emit Conflict Detection and Resolution information
to the log group. For information about the format of the log messages, see the documentation
for Conflict Detection and Sync Logging.
Sync Operations
Versioned data sources support Sync
operations that allow you to retrieve
all the results from a DynamoDB table and then receive only the data altered since your last
query (the delta updates). When AWS AppSync receives a request for a Sync
operation, it uses the fields specified in the request to determine if the Base table or the Delta table should be accessed.
-
If the
lastSync
field is not specified, aScan
on the Base table is performed. -
If the
lastSync
field is specified, but the value is before thecurrent moment - DeltaSyncTTL
, aScan
on the Base table is performed. -
If the
lastSync
field is specified, and the value is on or after thecurrent moment - DeltaSyncTTL
, aQuery
on the Delta table is performed.
AWS AppSync returns the startedAt
field to the response mapping template for
all Sync
operations. The startedAt
field is the moment, in epoch
milliseconds, when the Sync
operation started that you can store locally and
use in another request. If a pagination token was included in the request, this value will
be the same as the one returned by the request for the first page of results.
For information about the format for Sync
mapping templates, see the mapping
template reference.