Understanding the delivered files
AWS Marketplace writes everything under the AWSDataSubscriptions/ prefix of your bucket.
Each delivered file has the following key. In the key pattern, account-id
is your AWS account ID, dataset is the folder from Available datasets, and
the date is the delivery day in UTC.
AWSDataSubscriptions/account-id/dataset/yyyy/MM/dd/file
-
account-id– Your AWS account ID. -
dataset– The dataset folder name. -
yyyy/MM/dd– The delivery day in UTC.
The following folder structure shows a bucket that receives the Marketplace Usage dataset.
AWSDataSubscriptions/ access-check.json 111122223333/ marketplace_daily_usage_report/ 2026/11/19/ 20261119063015_0000_part_00.parquet 20261119063015_0001_part_00.parquet 20261119151414_0000_part_00.parquet _SUCCESS 2026/11/20/ _SUCCESS
Keep the following points in mind about the delivered files:
-
There is one folder for each delivery day. A day folder can contain several files.
-
File names start with a 14-digit identifier of the delivery run. The rest of the name can vary. Use the
_SUCCESSfile to find a day's files. -
Files are never changed after delivery. Corrections arrive as new rows in a later file or on a later day.
-
Rows are change records. Each row has a
row_typecolumn with the valueINSERT,UPDATE, orDELETE. Each row also has arow_modified_datecolumn with the UTC time of the change, and one or more key columns (marked "(key column)" in the dataset tables). Apply the rows to your copy of the data by key, in order ofrow_modified_date. A row can be delivered more than once. Applying the same row again has no effect. -
AWSDataSubscriptions/access-check.jsonis an empty test file that AWS Marketplace writes when you create a subscription. You can ignore it or expire it with a lifecycle rule. If you add a rule, scope it to that file so that it doesn't expire your delivered files.
The _SUCCESS file
When a delivery day is complete, AWS Marketplace writes one _SUCCESS file into the day
folder, after every data file of the day. Read a day folder only after its
_SUCCESS file exists. The leading underscore makes Amazon Athena, Apache Spark, and
Apache Hive treat the file as hidden. A table that points at the dataset folder doesn't
read it as data.
The file is a JSON object that lists the files of the day. The following example shows a day with three files.
{ "schemaVersion": 1, "dataset": "marketplace_daily_usage_report", "day": "2026-11-19", "status": "DATA", "objectCount": 3, "totalBytes": 5242880, "objects": [ { "key": "20261119063015_0000_part_00.parquet", "size": 2097152 }, { "key": "20261119063015_0001_part_00.parquet", "size": 1048576 }, { "key": "20261119151414_0000_part_00.parquet", "size": 2097152 } ], "deliveredAt": "2026-11-20T02:10:44Z" }
The file contains the following fields:
status-
DATAwhen the day has files, orNO_DATAwhen the day is complete and nothing changed. Every day gets a_SUCCESSfile, so a missing file always means the day isn't finished, never that nothing happened. objects,objectCount,totalBytes-
The complete list of the day's files, with each key relative to the day folder and its size in bytes. AWS Marketplace writes the
_SUCCESSfile once and doesn't change it, so this list is authoritative. Read exactly these files, and ignore any other object in the folder. dataset,day,deliveredAt-
The folder, the delivery day, and the time in UTC when AWS Marketplace wrote the file.
schemaVersion-
The version of this format, currently
1. Ignore fields that you don't recognize.
Apply days in order. A later day's _SUCCESS file can appear before an earlier
day's. Wait for the earlier day and apply it first. To react to deliveries instead of
polling, configure an Amazon S3 Event Notification or an Amazon EventBridge rule on your bucket for the
prefix AWSDataSubscriptions/ and the
suffix account-id/_SUCCESS.