Subscribe to and access a product - AWS Data Exchange User Guide
Subscribe to and access a product containing file-based dataSubscribe to and access a product containing APIsSubscribe to and access a product containing Amazon Redshift data setsSubscribe to and access a product containing Amazon S3 data accessSubscribe to and access a product containing AWS Lake Formation data sets (Preview)Viewing and downloading a data dictionary

Subscribe to and access a product

The following topics describe the process of subscribing to and accessing a product on AWS Data Exchange using the AWS Data Exchange console.

Note

By subscribing to a product, you agree that your use of the product is subject to the provider's offer terms including pricing information and data subscription agreement (DSA).

You also agree and acknowledge that AWS may share information about the transaction (including your payment terms and product usage metrics) with the respective seller, reseller, or underlying provider, as applicable, in accordance with the AWS Privacy Notice. AWS will issue invoices and collect payments from you on behalf of the provider through your AWS account. Your use of AWS services remains subject to the AWS Customer Agreement or other agreement with AWS governing your use of such services.

Subscribe to and access a product containing file-based data

The following topics describe the complete process of subscribing to and accessing a product containing file-based data stored as files on AWS Data Exchange. To complete the process, use the AWS Data Exchange console.

For information about how to evaluate a product before subscribing, see Evaluate products containing data dictionaries and samples.

The process has the following steps:

To practice subscribing to and accessing a product containing file-based data, see the Tutorial: Subscribe to AWS Data Exchange Heartbeat on AWS Data Exchange.

Step 1: Subscribing to a product containing the file-based data

If you subscribe to a paid product, you are billed on your AWS bill. You get access to all entitled data sets. For more information, see Subscribing to data products on AWS Data Exchange.

To subscribe to a product containing the file-based data

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left side navigation pane, under Discover data products, choose Browse catalog.

  3. Under Refine results, use the Data set type filter and select Files (Amazon S3 Objects) to find products containing file-based data.

    For more information, see Browse the catalog.

  4. Select a data product containing Files (Amazon S3 Objects), and view its product detail page.

    The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and durations, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under Similar products.

    If the provider has issued a custom offer to your account (for example, a private offer or Bring Your Own Subscription (BYOS) offer), you see those details, too.

  5. In the top right corner, choose Continue to subscribe.

  6. Choose your preferred price and duration combination, choose whether to enable auto-renewal for the subscription, and review the offer details, including the DSA.

    Note

    Some products require subscription verification. For more information, see Subscription verification for subscribers.

  7. Review the pricing information, choose the pricing offer, and then choose Subscribe.

    Note

    If you subscribe to a paid product, you are prompted to confirm your decision to subscribe.

  8. Under Data sets included with your subscription, view the listed Data sets.

    After the subscription finishes processing, you can choose a data set to access your entitled data or choose View subscription to view your subscription.

  9. (Optional) For Set up exports - optional, select the check boxes for the data sets that contain the revisions that you want to export. Selecting a data set will prepare its most recently published revision to be exported.

    1. Choose a Simple destination option to select an Amazon S3 bucket location or choose Advanced to configure an Amazon S3 key naming pattern. This choice determines where your revisions will be exported. For more information about using key patterns, see Key patterns when exporting revisions.

    2. For Auto-export future revisions, choose whether to turn on or turn off automatic revision export:

      • On – All future revisions will always be exported.

      • Off – Only one export of the most recent revision will be exported.

    3. Choose the Encryption options, and review the Amazon S3 pricing.

      Note

      If you choose to export using AWS Key Management Service (AWS KMS) encryption, make sure your account has the correct AWS Identity and Access Management (IAM) permissions to create and revoke grants on the AWS KMS key you choose. Without these permissions, automatic export will fail.

    4. Choose Export to export the data to Amazon S3, or choose Skip if you prefer to wait and export or download later. For more information about how to export data after subscribing, see (Optional) Exporting data.

      Note

      It can take a few minutes for your subscription to become active after you choose Subscribe. If you choose Export before the subscription is active, you are prompted to wait until it is complete.

      After your subscription is active, your export will begin.

      Navigating away from this page prior to your subscription becoming active will not prevent the subscription from processing. It will prevent your data export from occurring.

Step 2: Accessing a product containing file-based data

After you successfully subscribe to a product, you have access to the product data sets according to the terms of the data subscription agreement (DSA).

The following topic describes how to access a product containing file-based data.

Viewing data sets, revisions, and assets

To view the data sets, revisions, and assets

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left navigation pane, choose Subscriptions, and then choose your product.

  3. View the data sets that are part of the product under Entitled data sets.

  4. Choose a data set.

  5. View the Data set overview, Auto-export job destinations (Amazon S3 products only), the Revisions, and the Description of the data set.

    For more information, see Data in AWS Data Exchange.

  6. Choose a revision.

    Revisions are listed from latest to oldest.

  7. View the Revision overview, Assets, and the Jobs that have been performed.

    For information about exporting file-based assets, see Exporting assets to an S3 bucket as a subscriber (console).

(Optional) Exporting data

After your subscription is active, you can set up your Amazon S3 bucket to receive assets that you export.

You can export the associated assets to Amazon S3 or you can use jobs with a signed URL.

If you want to export or download your data at a later time, including getting new revisions, see Exporting assets to an S3 bucket as a subscriber (console).

Important

We recommend that you consider Amazon S3 security features when exporting data to Amazon S3. For more information about general guidelines and best practices, see Security best practices for Amazon S3 in the Amazon Simple Storage Service User Guide.

For more information about how to export data, see Exporting assets and Exporting revisions.

Subscribe to and access a product containing APIs

The following topics describe the complete process of subscribing to and accessing a product containing APIs on AWS Data Exchange by using the AWS Data Exchange console.

For information about how to evaluate a product before subscribing, see Evaluate products containing data dictionaries and samples.

The process has the following steps:

To practice subscribing to and accessing a product containing APIs, see the Tutorial: Subscribe to AWS Data Exchange for APIs (Test Product) on AWS Data Exchange.

Step 1: Subscribing to a product containing APIs

If you subscribe to a paid product, you're billed on your AWS bill. You get access to all entitled data sets. For more information, see Subscribing to data products on AWS Data Exchange.

A provider might include metered costs to their product containing APIs. If a provider decreases metered costs, the price decrease goes into effect immediately. If the provider increases metered costs, and you're an existing subscriber, the price increase goes into effect on the first day of the month, 90 days after the price increase was submitted OR upon renewal (whichever is sooner). An email message is sent to existing subscribers when the price change is submitted.

For example, assume that a provider submits a metered cost price increase on May 10. Existing subscribers receive an email message about the price change. The price increase goes into effect on September 1.

To subscribe to a product containing APIs

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left side navigation pane, under Discover data products, choose Browse catalog.

    For more information, see Browse the catalog.

  3. Under Refine results, use the Data set type filter and select API to find products containing APIs.

    For more information, see Browse the catalog.

  4. Select a product containing APIs, and view its product detail page.

    The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and durations, metered costs (if included), the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under Similar products.

    If the provider has issued a custom offer to your account (for example, a private offer or Bring Your Own Subscription (BYOS) offer), you see those details, too.

    1. Under Public offer, view the API metered costs (if included).

    2. (Optional) In the Metered cost calculator, choose Select metered cost and then enter the number of units to display an example of the cost.

  5. In the top right corner, choose Continue to subscribe.

  6. Choose your preferred price and duration combination, choose whether to enable auto-renewal for the subscription, and review the offer details, including the DSA.

    Note

    Some products require subscription verification. For more information, see Subscription verification for subscribers.

  7. Review the pricing information, choose the pricing offer, and then choose Subscribe.

    Note

    If you subscribe to a paid product, you're prompted to confirm your decision to subscribe.

  8. Under Data sets included with your subscription, view the listed Data sets.

    After the subscription finishes processing, you can choose a data set to access your entitled data or choose View subscription to view your subscription.

Step 2: Accessing an API product

The following topics provide details about how to access a product that includes API data sets:

Viewing an API

To view an API

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left navigation pane, under My subscriptions, choose Entitled data.

  3. Choose a data set.

  4. Under the Revisions tab, choose a revision.

  5. Under API assets, choose the API.

  6. View the Asset overview.

  7. Follow the guidance in the Integration notes to call the API.

Downloading the API specification

To download the API specification

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left navigation pane, under My subscriptions, choose Entitled data.

  3. Choose a data set.

  4. Under the Revisions tab, choose a revision.

  5. Under API assets, choose the API.

  6. On the OpenAPI 3.0 specification, choose Download API specification.

    The specification is downloaded onto your local computer. You can then export the asset to a third-party tool for SDK generation.

Making an API call (console)

You can call a single endpoint in the AWS Data Exchange console.

To make an API call from the console

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left navigation pane, under My subscriptions, choose Entitled data.

  3. Choose a data set.

  4. Under the Revisions tab, choose a revision.

  5. Under API assets, choose the API.

  6. For Integration notes:

    1. Choose Copy to use the Base URL.

    2. Choose Copy to use the Code structure.

    3. Follow the information provided in the specification documentation to call the API.

Making an API call (AWS CLI)

To make an API call (AWS CLI)

  • Use the send-api-asset command to call the API.

    $ aws dataexchange send-api-asset \
  --asset-id $ASSET_ID \
  --data-set-id $DATA_SET_ID \
  --revision-id $REVISION_ID \
  --body "..." \
{
    "headers": {
        ...
    },
    "body": "..."
}

Subscribe to and access a product containing Amazon Redshift data sets

Overview for subscribers

An Amazon Redshift data set is a data set that contains AWS Data Exchange datashares for Amazon Redshift. Datashares give you read-only access to the tables, views, schemas, and user-defined functions that a data provider adds to the datashare.

As a data subscriber, you can find and subscribe to products containing Amazon Redshift data sets. After your subscription starts, you get access to query the data in Amazon Redshift without extracting, transforming, and loading data. You lose access to a product's datashares after your subscription expires.

Consider the following:

  • It might take a few minutes to access the datashares after your subscription starts.

The following sections describe the complete process of becoming an Amazon Redshift datashare product subscriber on AWS Data Exchange by using the AWS Data Exchange console.

For information about how to evaluate a product before subscribing, see Evaluate products containing data dictionaries and samples.

The process has the following steps:

To practice subscribing to and accessing a product containing Amazon Redshift data sets, see the Tutorial: Subscribe to Worldwide Event Attendance (Test Product) on AWS Data Exchange.

Step 1: Subscribing to products containing Amazon Redshift data sets

If you subscribe to a paid product, you're billed on your AWS bill. You get access to all data sets included in the product. For more information, see Subscribing to data products on AWS Data Exchange.

To subscribe to a product containing Amazon Redshift data sets

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left side navigation pane, under Discover data products, choose Browse catalog.

    For more information, see Browse the catalog.

  3. Under Refine results, use the Data set type filter and select Amazon Redshift to find products containing Amazon Redshift datashares.

    For more information, see Browse the catalog.

  4. Select a product and view its product detail page.

    The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and duration, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under Similar products.

    If the provider has issued a custom offer to your account (for example, a private offer or Bring Your Own Subscription (BYOS) offer), you see those details, too.

    Important

    Be sure to review the date, time, and duration of the cluster’s maintenance window. During the maintenance window, you do not have access to the datashare.

  5. In the top right corner, choose Continue to subscribe.

  6. Review the Product offer, the Subscription terms, the Data sets that are included in the offer, and the Support information.

  7. Choose whether to enable Offer auto-renewal for the subscription

    Note

    Some products require subscription verification. For more information, see Subscription verification for subscribers.

  8. Choose Subscribe.

    Note

    If you subscribe to a paid product, you're prompted to confirm your decision to subscribe.

  9. Under Data sets included with your subscription, view the listed Data sets.

    After the subscription finishes processing, you can choose a data set to access your entitled data or choose View subscription to view your subscription.

Step 2: Accessing the AWS Data Exchange datashares for Amazon Redshift

You have access to the product's data sets according to the terms of the data subscription agreement (DSA). As a subscriber, your subscription to a product that includes AWS Data Exchange datashares for Amazon Redshift gives you read-only access to the tables, views, schemas, and functions within the datashare.

With a subscription, you can do the following:

  • Query data without having to extract, transform, or load data.

  • Access the latest provider data as soon as the provider updates it.

For more information, see Working with AWS Data Exchange datashares in the Amazon Redshift Database Developer Guide.

Note

You lose access to a product's datashares after your subscription expires.

For more information about how to subscribe to an Amazon Redshift data set, see Tutorial: Subscribe to Worldwide Event Attendance (Test Product) on AWS Data Exchange.

Subscribe to and access a product containing Amazon S3 data access

Overview for subscribers

AWS Data Exchange for Amazon S3 allows data subscribers to access third-party data files directly from data providers' Amazon S3 buckets.

As a data subscriber, after you are entitled to an AWS Data Exchange for Amazon S3 data set, you can start your data analysis with AWS services such as Amazon Athena, SageMaker Feature Store, or Amazon EMR directly using the provider's data in their Amazon S3 buckets.

Consider the following:

  • Providers have the option to enable Requester Pays, an Amazon S3 feature, on the Amazon S3 bucket hosting the data offered. If enabled, subscribers pay to read, use, transfer, export, or copy data into their Amazon S3 buckets. For more information, see Using Requester Pays buckets for storage transfers and usage in the Amazon Simple Storage Service User Guide.

  • When you subscribe to an AWS Data Exchange for Amazon S3 data product, AWS Data Exchange automatically provisions an Amazon S3 access point and updates its resource policies to grant you read-only access. Amazon S3 access points is a feature of Amazon S3 that simplifies data sharing to an Amazon S3 bucket. For more information, see Managing data access with Amazon S3 access points in the Amazon Simple Storage Service User Guide.

  • Before you use the Amazon S3 access point Amazon Resource Name (ARN) or alias to access the shared data, you must update your IAM permissions. You can verify that the current role and its associated policy allows GetObject and ListBucket calls to the provider’s Amazon S3 bucket and the Amazon S3 access point provided by AWS Data Exchange.

The following sections describe the complete process of becoming an AWS Data Exchange for Amazon S3 subscriber by using the AWS Data Exchange console.

The process has the following steps:

Step 1: Subscribing to products containing Amazon S3 data access

If you subscribe to a paid product, you're billed on your AWS bill. You get access to all data sets included in the product. For more information, see Subscribing to data products on AWS Data Exchange.

To subscribe to a product containing access to Amazon S3

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left side navigation pane, under Discover data products, choose Browse catalog.

    For more information, see Browse the catalog.

  3. Under Refine results, use the Data set type filter and select Access to Amazon S3 to find products containing access to Amazon S3 data.

    For more information, see Browse the catalog.

  4. Select a product and view its product detail page.

    The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and duration, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they are available. You can also continue to browse other product detail pages by choosing a product under Similar products.

    If the provider has issued a custom offer to your account (for example, a private offer or Bring Your Own Subscription (BYOS) offer), you see those details, too.

  5. In the top right corner, choose Continue to subscribe.

  6. Review the Product offer, the Subscription terms, the Data sets that are included in the offer, and the Support information.

  7. Choose whether enable Offer auto-renewal for the subscription

    Note

    Some products require subscription verification. For more information, see Subscription verification for subscribers.

  8. Choose Subscribe.

    Note

    If you subscribe to a paid product, you're prompted to confirm your decision to subscribe.

  9. Under Data sets included with your subscription, view the listed Data sets.

    After the subscription finishes processing, you can choose a data set to access your entitled data or choose View subscription to view your subscription.

Step 2: Accessing a product containing Amazon S3 data access

You can run queries to analyze the data in-place without setting up your own Amazon S3 buckets, copying data files into Amazon S3 buckets, or paying associated storage fees. You access the same Amazon S3 objects that the data provider maintains allowing you to use the most current data available.

With a subscription, you can do the following:

  • Analyze data without setting up individual Amazon S3 buckets, copying files, or paying storage fees.

  • Access the latest provider data as soon as the provider updates it.

To view the data sets, revisions, and assets

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left navigation pane, under My subscriptions, choose Entitled data.

  3. On the Entitled data page, expand a product and choose a data set.

  4. View the Data set overview.

    Note

    The data provided is stored in the provider's Amazon S3 bucket. When accessing this data, you’ll be responsible for the cost of the request and the data downloaded from the provider’s Amazon S3 bucket, unless the provider specifies otherwise.

  5. Before getting started, your role must have IAM permissions to use your entitled Amazon S3 data access. On the Data set overview page, on the Amazon S3 data access tab, select Verify IAM permissions to determine if your role has the correct permissions to access your data.

  6. If you have the necessary IAM permissions, choose Next on the IAM Policy prompt displayed. If you don't have the needed permissions, follow the prompt to embed the JSON policy in the user or role.

  7. Review your Shared locations to view the Amazon S3 bucket or prefixes and objects shared by the provider. Review the data access information for Amazon S3 Access Point information to determine if the provider enabled Requester Pays.

  8. Choose Browse shared Amazon S3 locations to view and explore the provider's Amazon S3 bucket, prefixes, and objects shared.

  9. Use the Access Point alias anywhere you use Amazon S3 bucket names to access your entitled data programmatically. For more information, see Using access points with compatible Amazon S3 operations in the Amazon Simple Storage Service User Guide.

  10. (Optional) When you gain an entitlement to an Amazon S3 data access data set that contains data encrypted with a provider’s AWS KMS key, you can view the KMS key ARN in your subscriber console. AWS Data Exchange creates an AWS KMS grant on the key for you, so you can access the encrypted data. You must obtain kms:Decrypt IAM permission on the KMS key to read encrypted data from the Amazon S3 Access Point from which you’ve gained entitlement. You can choose between the following IAM policy statements:

    1. IAM policy allowing users to decrypt or encrypt data with any KMS key.

      
{
     "Version": "2012-10-17",
     "Statement": [
         {
              "Effect": "Allow",
              "Action": [
                 "kms:Decrypt"
              ],
              "Resource": [
                  "*"
              ]
          }
     ]
}

    2. IAM policy allowing users to specify the exact KMS key ARNs visible in the subscriber console.

      
{    
    "Version": "2012-10-17",
    "Statement": [
        {
             "Effect": "Allow",
             "Action": [
                "kms:Decrypt"
             ],
             "Resource": [
                 "<KMS key Arn from subscriber's console>  
             ]
        }
    ]
}
    Note

    AWS KMS grants can take up to 5 minutes for the operation to achieve eventual consistency. You might not have access to the Amazon S3 data access data set until this is complete. For more information, see Grant in AWS KMS in the AWS Key Management Service Developer Guide.

For more information about how to subscribe to an Amazon S3 data set, see Subscribe to and access a product containing Amazon S3 data access.

Subscribe to and access a product containing AWS Lake Formation data sets (Preview)

Overview for subscribers

An AWS Lake Formation data set is a data set that contains AWS Lake Formation data permission assets.

As a data subscriber, you can find and subscribe to products containing AWS Lake Formation data sets. Once you're entitled to an AWS Data Exchange for AWS Lake Formation data set, you can query, transform, and share access to the data within your AWS account using AWS Lake Formation, or across your AWS organization using AWS License Manager.

Step 1: Subscribing to products containing AWS Lake Formation data sets

If you subscribe to a paid product, you're billed on your AWS bill. You get access to all data sets included in the product. For more information, see Subscribing to data products on AWS Data Exchange.

To subscribe to a product containing AWS Lake Formation data sets

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left side navigation pane, under Discover data products, choose Browse catalog.

    For more information, see Browse the catalog.

  3. Under Refine results, use the Data set type filter and select AWS Lake Formation to find products containing AWS Lake Formation data sets.

  4. Select a product and view its product detail page.

    The information on the product detail page includes a product description, the provider's contact information, and the details of the product's public offer. The public offer information includes price and duration, the data subscription agreement (DSA), and the refund policy. You can view the names of the data sets included in the product and the AWS Regions in which they're available. You can also continue browsing other product detail pages by choosing a product under Similar products.

    If the provider has issued a custom offer to your account (for example, a private offer or Bring Your Own Subscription (BYOS) offer), you see those details, too.

  5. In the top right corner, choose Continue to subscribe.

  6. Review the Product offer, the Subscription terms, the Data sets that are included in the offer, and the Support information.

  7. Choose whether to enable Offer auto-renewal for the subscription.

    Note

    Some products require subscription verification. For more information, see Subscription verification for subscribers.

  8. Choose Subscribe. If you subscribe to a paid product, you're prompted to confirm your decision to subscribe.

  9. Under Data sets included with your subscription, view the listed Data sets.

    After the subscription finishes processing, you can choose a data set to access your entitled data or choose View subscription to view your subscription.

Step 2: Accessing the AWS Data Exchange datashares for AWS Lake Formation

After you subscribe to a product containing AWS Lake Formation data sets, you can use Lake Formation compatible query engines, like Amazon Athena, to query your data.

After subscription completion, you must do the following:

  1. Accept the AWS Resource Access Manager (AWS RAM) share within 12 hours after you subscribe to the product. You can accept the AWS RAM share from your subscription page or the entitled data page for your AWS Lake Formation data permission data set on the AWS Data Exchange console. You only need to accept an AWS RAM share once per provider. For more information about accepting a resource share invitation from AWS RAM, see Accepting a resource share invitation from AWS RAM.

  2. Navigate to AWS Lake Formation and create resource links from the new shared resources.

  3. Navigate to Athena or another AWS Lake Formation compatible query engine to query your data.

Viewing and downloading a data dictionary

Providers can attach data dictionaries to all products. The following procedures describe how to view and download a data dictionary.

For more information about data dictionaries and samples, see Data dictionaries and samples.

To view and download a data dictionary

  1. Open and sign in to the AWS Data Exchange console.

  2. In the left navigation pane, under My subscriptions, choose Entitled data.

  3. On the Entitled data page, expand a product and choose a data set.

  4. On the data set details page, choose the Data dictionary tab.

    • View the data dictionary on the Data dictionary tab.

    • Download the data dictionary by choosing Download and then saving the file to your computer.