Tutorial: Subscribe to AWS Data Exchange for Amazon S3 (Test Product)

AWS Data Exchange for Amazon S3 (Test Product) is a product that helps subscribers understand how to subscribe to and interact with an AWS Data Exchange product. In this tutorial, the product contains Amazon Simple Storage Service (Amazon S3) data access data sets.You can use this product for testing purposes and to learn how to query and analyze data directly from a data provider's Amazon S3 bucket.

You can run queries to analyze the data in-place without setting up your own Amazon S3 buckets, copying data files into Amazon S3 buckets, or paying associated storage fees.

Subscribing to AWS Data Exchange for Amazon S3 (Test Product)

The following procedure shows how to browse the AWS Data Exchange catalog to find and subscribe to AWS Data Exchange for Amazon S3 (Test Product).

To find and subscribe to AWS Data Exchange for Amazon S3 (Test Product)

1. Open and sign in to the AWS Data Exchange console.

2. From the left navigation pane, under Discover data products, choose Browse catalog.

3. From the search bar, enter AWS Data Exchange for Amazon S3 (Test Product) and choose Enter.

4. Choose AWS Data Exchange for Amazon S3 (Test Product) to view its details page.

   1. (Optional) To view the data dictionary, scroll down to the product Overview section to see the data dictionary under Data dictionaries.

   2. (Optional) To download the data dictionary, choose the Data dictionary and samples tab, choose the option button next to Data dictionary, and then choose Download.

   3. (Optional) To preview the sample, choose the option button next to the sample name of Blockchain Transactions (Test Data), and then choose Preview sample (CSV only).

   4. (Optional) To download the sample, choose the option button next to the sample name of Blockchain Transactions (Test Data), and then choose Download.

      If you're previewing the sample, you can also choose Download in the sample preview dialog box.

5. In the top right corner, choose Continue to subscribe.

6. Choose your preferred price and duration combination. Choose whether to enable auto-renewal for the subscription, and review the offer details, including the data subscription agreement.

   Note

   AWS Data Exchange for Amazon S3 (Test Product) doesn't require subscription verification, but some products do. For more information, see Subscription verification for subscribers.

7. Review the pricing information, choose the pricing offer, and then choose Subscribe.

   Note

   AWS Data Exchange for Amazon S3 (Test Product) is a free product. If you subscribe to a paid product, you are prompted to confirm your decision to subscribe.

It can take a few minutes for your subscription to become active after you choose Subscribe.

Navigating away from this page before your subscription becomes active will not prevent the subscription from processing.

Setting up and querying AWS Data Exchange for Amazon S3 (Test Product)

The following procedure shows how to set up and query an Amazon S3 data access data set using the AWS Command Line Interface (AWS CLI). Before querying, you must obtain the appropriate AWS Identity and Access Management (IAM) permissions to attach policies to your user. To access data in a provider's bucket directly through the Amazon S3 delivery method, embed the following JSON policy to the user or role.

To set up AWS Data Exchange for Amazon S3 (Test Product)

1. Open and sign in to the AWS Data Exchange console.

2. From the left navigation pane under My subscriptions, choose Entitled data in the AWS Region that hosts the data set. For the purposes of this tutorial, the Region is us-east-1.

3. From the list of Products, choose AWS Data Exchange for Amazon S3 (Test Product) and then choose the Blockchain Transactions (Test Data) data set.

4. Choose Verify IAM permissions.

   Note

   If you don't have the correct permissions, you'll receive a notification detailing how to create and attach the IAM policy to your user or role. In the following example, replace each user input placeholder with your own information.

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": "s3:ListBucket",
         "Resource": [
           "arn:aws:s3:us-east-1:111122223333:accesspoint/my-access-point",
           "arn:aws:s3:::aws-data-exchange-s3-data-access-btc-demo-us-east-1"
         ]
       },
       {
         "Effect": "Allow",
         "Action": "s3:GetObject",
         "Resource": [
           "arn:aws:s3:us-east-1:111122223333:accesspoint/my-access-point/object/*",
           "arn:aws:s3:::aws-data-exchange-s3-data-access-btc-demo-us-east-1/*"
         ]
       }
     ]
   }

To allow querying on the AWS Data Exchange for Amazon S3 (Test Product) data access data set using the AWS CLI

1. Open AWS CloudShell in us-east-1.

2. Choose the copy button next to the access point alias to copy and paste the code inside. After the command is added inAWS CloudShell with the correct access point alias, you can see the list of Amazon S3 objects included in this product.

   Important

   When a provider has enabled Requester Pays, the subscriber pays for the data transfer and the request. The provider pays for the data storage. For more information, see Using Requester Pays buckets for storage transfers and usage in the Amazon Simple Storage Service User Guide.

3. (Optional) You can also copy an object to your local system using the following command.

   aws s3api get-object --bucket <Access point alias> --key 'v1.0/btc/transactions/date=2022-11-27/part-00000-03a88dba-27dd-4f59-a890-70a3d2c7ad26-c000.snappy.parquet' aws_btc.snappy.parquet --request-payer requester