Publishing a new product - AWS Data Exchange User Guide

Publishing a new product

The following topics describe the process of publishing a new product on AWS Data Exchange by using the AWS Data Exchange console.

The following video explains more about how to publish a new data product on AWS Data Exchange.

Publishing a product containing file-based data

The following topics describe the process of creating a data set and publishing a new product containing file-based data on AWS Data Exchange by using the AWS Data Exchange console. The process has the following steps:

Step 1: Create assets

Assets are the data in AWS Data Exchange. For more information, see Assets.

Before you create and publish a new file-based data product, you must:

  1. Create your files.

    AWS Data Exchange supports all file types.

  2. Store your files as objects in Amazon Simple Storage Service (Amazon S3) or on your local computer.

    For more information about storing files in Amazon S3, see the Amazon S3 User Guide.

Step 2: Create a data set

Data sets in AWS Data Exchange are dynamic and are versioned using revisions, with each revision containing at least one asset. For more information, see Data in AWS Data Exchange.

To create a data set
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. In the left side navigation pane, under Publish data, choose Owned data sets.

  3. In Owned data sets, choose Create data set to open the Data set creation steps wizard.

  4. In Select data set type, choose Files.

  5. In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.

  6. (Optional) Under Add tags – optional, add tags.

  7. Choose Create data set.

Step 3: Create a revision

In the following procedure, you create a revision after you’ve created a data set in the AWS Data Exchange console. For more information, see Revisions.

To create a revision
  1. On the Data set overview section of the data set details page:

    1. (Optional) Choose Edit name to edit information about your data set.

    2. (Optional) Choose Delete to delete the data set.

  2. In the Revisions section, choose Create revision.

  3. Under Define revision, provide an optional comment for your revision that describes the purpose of the revision.

  4. (Optional) Under Add tags – optional, add tags associated with the resource.

  5. Choose Create revision.

  6. Review, edit, or delete your changes from the previous step.

Step 4: Import assets to a revision

In the following procedure, you import data assets, and then finalize the revision in the AWS Data Exchange console. For more information, see Assets.

To import assets to the revision
  1. Under the Jobs section of the data set details page, choose either Import from Amazon S3 or Upload (to upload from your computer), depending on where the data assets for the data set are currently stored.

  2. Follow the prompts, depending on your selection. A job is started to import your asset into your data set.

  3. After the job is finished, the State field in the Jobs section is updated to Completed.

  4. If you have more data to add, repeat Step 1.

  5. In Revision overview, review your revision and its assets.

  6. Choose Finalize revision.

You have successfully finalized a revision for a data set.

You can edit or delete a revision before you add it to a product.

Edit a revision

To edit the revision after you’ve finalized it
  1. In Revision overview, choose De-finalize.

    You see a message that the revision is no longer in the finalized state.

  2. To edit the revision, from Revision overview, choose Actions, Edit.

  3. Make your changes, and then choose Update.

  4. Review your changes, and then choose Finalize.

Delete a revision

To delete the revision after you’ve finalized it
  1. In Revision overview, choose Delete.

  2. Type Delete in the Delete revision dialog box, and then choose Delete.

Warning

This deletes the revision and all of its assets. This action cannot be undone.

Step 5: Publish a new product

After you've created at least one data set and finalized a revision with assets, you're ready to publish that data set as a part of a product. For more information, see Product details. Make sure that you have all required details about your product and offer.

To publish a new product
  1. In the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.

  2. From Products, choose Publish new product to open the Publish new product wizard.

  3. In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.

  4. In the Add data section, under Owned data sets, select the check boxes next to the data sets you want to add, and then choose Add selected.

    Note

    The data sets you choose must have a finalized revision. Data sets without finalized revisions can't be added.

    1. Go to Selected data sets to review your selections.

      You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.

    2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next. For more details, see Revision access rules.

  5. In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

    For more information, see Product details.

  6. (Optional) In the Define product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

    1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.

      You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.

    2. Choose a saved data dictionary from your computer, and then choose Open.

      The data dictionary .csv file appears on the Edit dialog box.

      Note

      Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.

    3. Choose Data dictionary preview to preview it.

    4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.

      The samples appear on the Edit dialog box.

      Note

      You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.

    5. Enter a description for each sample that will be visible on the product detail page.

    6. Choose Save.

  7. Under Product definition, enter a Short description and a Long description of your product.

    If you want to use a template for your long description, select Apply template, choose your template type, and then fill out the template with your specific product details.

  8. Choose Next.

  9. Configure your offer.

    • If you're creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.

      1. Choose your Pricing and access duration options for the subscription.

      2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      3. (Optional) Set Subscription verification, which enables you to control who can subscribe to this product. For more information, see Subscription verification for providers.

      4. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      5. Choose Next.

    • If you're creating a private offer, configure the offer details in the Add custom offer section.

      1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.

      2. Choose your Pricing and access duration options for the subscription.

      3. Choose the Offer expiration date by which the subscriber must accept the offer.

      4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      5. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      6. Choose Next.

  10. In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.

  11. If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval and then changes to Published after it's published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom offers associated with the product will not be copied, but public offers will be copied.

To copy a product
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. From the left navigation pane, under Publish data, choose Products.

  3. From Products, choose the button next to the product you want to copy.

  4. Select the Actions dropdown, and then choose Create copy.

  5. Continue through the Publish a new product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing APIs

Overview

The following topics describe the process of creating a REST API data set and publishing a new product that contains APIs on AWS Data Exchange. You can complete the process by using either the AWS Data Exchange console or the AWS Command Line Interface.

After you have set up your Amazon API Gateway REST API, you can create a new API data set in AWS Data Exchange. You can then create a revision, and add API assets.

Creating and publishing an API asset allows subscriber requests to an AWS Data Exchange endpoint to proxy through to your API Gateway API. You can then add this data set to a product and add pricing. Then, subscribers can view your product and subscribe to it in the AWS Marketplace catalog and the AWS Data Exchange catalog.

AWS Data Exchange features are available including revision access rules, private products, private offers, and subscription verification.

You can choose only contract-based pricing, metered cost pricing (where the contract pricing is $0), or a combination of metered and contract pricing.

You can choose standard metered costs, or you can specify a custom metered cost. There are three types of standard metered costs available:

  • Per API request

  • Per successful API request

  • Per unit of data transferred in bytes

Note

Metered costs apply to all API data sets in a product. Therefore, if you want to charge different prices for the same dimension for different API data sets, we recommend that you create these data sets in different products.

The process has the following steps:

Prerequisites

Before you can publish a product containing APIs, you must meet the following prerequisites:

  • Before you can use any AWS service, including AWS Data Exchange, you must sign up for AWS and create an AWS Identity and Access Management (IAM) user account. For more information, see Setting up AWS Data Exchange.

  • To create products on AWS Data Exchange, you must register your AWS account as an AWS Marketplace Seller. Use this account to create your data sets. The account with the API Gateway resource doesn't need to be in the same account that is creating the data sets.

  • Your REST API must be on Amazon API Gateway with an integration that uses an appropriate request and response model for accessing your data, such as Amazon DynamoDB or AWS Lambda. For more information, see Developing a REST API in API Gateway and Working with REST APIs in the Amazon API Gateway Developer Guide.

    Note

    Only public API Gateway APIs are supported.

  • Your API Gateway REST API must be able to authenticate and authorize calls from the AWS Data Exchange service principal. Every request from AWS Data Exchange to your API uses the Signature Version 4 (SigV4) protocol signed with AWS Data Exchange credentials. AWS Data Exchange works with custom domains and domain key mappings.

    Note

    AWS Data Exchange doesn't support Amazon Cognito, No-Auth, and AWS Lambda authorizers.

  • If your API Gateway REST API uses a custom identity system for authentication and authorization, configure it to use IAM authentication and import an OpenAPI schema describing your API. AWS Data Exchange will invoke your API Gateway REST API with its own service credentials and include subscriber information such as account ID.

  • Your API Gateway REST API is responsible for integrating with your backend. To do this, do one of the following:

    • Attach a long-lived authentication token to every request that comes through your API Gateway REST API that the backend can verify.

    • Use API Gateway to invoke a Lambda function that can generate credentials and invoke your API.

Your API is invoked per the API integration request specification.

For more information, see the following topics:

API data set security

AWS Data Exchange encrypts traffic end to end using Transport Layer Security (TLS) 1.2. All metadata is encrypted at rest. AWS Data Exchange will not store subscriber requests or the responses from your backend. We only extract metering metadata necessary for billing.

API integration request specification

An API on AWS Data Exchange passes through all headers (except for the headers listed in Header forwarding), body, http method, path, and query strings as-is from the customer request and appends the following headers.

// These headers help prevent Confused Deputy attacks. They enable the SourceAccount // and SourceArn variables in IAM policies. 'x-amz-source-account': ACCOUNT_ID, 'x-amz-source-arn': `arn:aws:dataexchange:${REGION}:${OWNER_ACCOUNT_ID}:data-sets/${DATA_SET_ID}/revisions/${REVISION_ID}/assets/${ASSET_ID}`, // These headers identify the API Asset in Data Exchange. 'x-amzn-dataexchange-asset-id': ASSET_ID, 'x-amzn-dataexchange-data-set-id': DATA_SET_ID, 'x-amzn-dataexchange-revision-id': REVISION_ID, // This header identifies the Data Exchange Product. 'x-amzn-dataexchange-product-id': PRODUCT_ID, // This header identifies the caller of Data Exchange. It will contain subscriber // information. 'x-amzn-dataexchange-requester-account-id': REQUESTER_ACCOUNT_ID, // Providers can attach custom metadata in the form of key/value pairs // to a particular subscription. We will send these key/value pairs as stringified // JSON. 'x-amz-dataexchange-subscription-metadata': STRINGIFIED_METADATA,
Header forwarding

AWS Data Exchange removes any headers related to authentication or namespaced to Amazon prior to forwarding it to a provider backend. Specifically, AWS Data Exchange removes:

  • Authentication header

  • Any headers that begin with x-amz

The host header will be overwritten as a consequence of the proxying.

Step 1: Update the API resource policy

If you have an Amazon API Gateway REST API that meets the Prerequisites, you must update your API resource policy to grant AWS Data Exchange the ability to invoke your API when a subscriber makes a request to get your API’s schema.

To update your API resource policy
  1. Add the following policy to your API’s resource policy:

    { "Effect": "Allow", "Principal": {"Service": "dataexchange.amazonaws.com"}, "Action": "execute-api:Invoke", "Resource": "*", "Condition": {"StringEquals": {"aws:SourceAccount": "<account-id>"}} }
  2. Replace account-id with the account that will be creating the API data set.

    The account with the API Gateway resource does not need to be in the same account that is creating the data set.

This policy restricts these permissions to calls made by the AWS Data Exchange service principal and requires that only your account can authorize AWS Data Exchange to integrate with your API.

Note

If you have a resource policy that explicitly denies AWS Data Exchange from doing this invocation, you must remove or limit this deny.

You’re now ready to create an API data set.

Step 2: Create an API data set

Data sets in AWS Data Exchange are dynamic and are versioned using revisions, with each revision containing at least one asset. For more information, see Data in AWS Data Exchange.

You use either the AWS Data Exchange console or the AWS Command Line Interface to create an API data set:

Creating an API data set (console)
To create an API data set (console)
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. On the left side navigation pane, under Publish data, choose Owned data sets.

  3. In Owned data sets, choose Create data set to open the Data set creation steps wizard.

  4. In Select data set type, choose Amazon API Gateway API.

  5. In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.

  6. (Optional) Under Add tags – optional, add tags.

  7. Choose Create.

You are now ready to create a revision.

Creating an API data set (AWS CLI)
To create an API data set (CLI)
  1. Use the create-data-set command to create an API data set:

    $ aws dataexchange create-data-set \ --asset-type API_GATEWAY_API \ --description 'Data Set Description' \ --name 'Data Set Name' { "Arn": "arn:aws:dataexchange:us-east-1:123456789012:data-sets/$DATA_SET_ID", "AssetType": "API_GATEWAY_API", "CreatedAt": "2021-09-11T00:16:46.349000+00:00", "Description": "Data Set Description", "Id": "$DATA_SET_ID", "Name": "Data Set Name", "Origin": "OWNED", "UpdatedAt": "2021-09-11T00:16:46.349000+00:00" }
  2. Note the new Asset Type of API_GATEWAY_API.

You are now ready to create a revision.

Step 3: Create a revision

In the following procedure, you create a revision after you’ve created a data set. For more information, see Revisions.

You use either the AWS Data Exchange console or the AWS Command Line Interface to create a revision:

Creating a revision (console)
To create a revision (console)
  1. On the Data set overview section of the data set details page:

    1. (Optional) Choose Edit name to edit information about your data set.

    2. (Optional) Choose Delete to delete the data set.

  2. On the Revisions section, choose Create revision.

  3. Under Define revision, provide an optional comment for your revision that describes the purpose of the revision.

  4. (Optional) Under Add tags – optional, add tags associated with the resource.

  5. Choose Create revision.

  6. Review, edit, or delete your changes from the previous step.

You are now ready to add API assets to the revision.

Creating a revision (AWS CLI)
To create a revision (AWS CLI)
  1. Use the create-revision command to create a revision:

    $ aws dataexchange create-revision \ --data-set-id $DATA_SET_ID \ --comment 'First Atlas Revision' { "Arn": "arn:aws:dataexchange:us-east-1:123456789012:data-sets/$DATA_SET_ID/revisions/$REVISION_ID", "Comment": "First Atlas Revision", "CreatedAt": "2021-09-11T00:18:49.160000+00:00", "DataSetId": "$DATA_SET_ID", "Finalized": false, "Id": "$REVISION_ID", "UpdatedAt": "2021-09-11T00:18:49.160000+00:00" }
  2. Add the API assets to the revision.

    Note

    You will need to know the ID of the API Gateway REST API you want to import as well as the stage.

Step 4: Add API assets to a revision

API assets contain the information subscribers need to make calls to your API. For more information, see Assets.

In the following procedure, you import data assets, and then finalize the revision.

You use either the AWS Data Exchange console or the AWS CLI to add assets to a revision:

Adding API assets to a revision (console)
To add assets to the revision (console)
  1. Under the API assets section of the data set details page, choose Add API stage.

  2. Under Select API stage, for Amazon API Gateway API, enter an API in the input box or choose one of the following from the drop-down list:

    • API in another AWS account – this is a cross account API that you have been given permission to access.

    • In this AWS account – this is an API in your AWS account.

    1. If you chose API in another AWS account, enter the API ID and the API Stage name in the input boxes.

    2. If you chose In this AWS account, choose the API Stage name from the drop-down list

    Note

    You can create a new API stage by choosing Create new and following the steps in the Create new API on Amazon API Gateway modal. Once the new stage has been created, repeat Step 2.

  3. Under Advanced configuration – optional, you can choose to Connect existing Amazon API Gateway usage plan to use the throttling and quota limits as defined in the existing usage plan, and enter the API key.

  4. Under Document API for subscribers, provide details about the API that the subscribers will see after they subscribe to your product.

    1. For API name, enter a name that subscribers can use to identify the API asset.

      Note

      If an In this AWS account was selected, the API name is automatically populated, which you can modify if necessary.

      If a API in another AWS account was selected, the API name is populated with a default name, which you should modify to so the subscriber can easily understand what it is.

    2. For OpenAPI 3.0 specification, either:

      1. Enter or copy and paste the OpenAPI 3.0 specification file.

      2. Choose Import from .JSON file, and then select the .json file from your local computer to import.

        The imported specification appears in the box.

      3. Choose Import from Amazon API Gateway, and then choose a specification to import.

        The imported specification appears in the box.

    3. For Additional documentation - optional, enter any additional information that is useful for the subscriber to know about your API. Markdown is supported.

    Note

    You can't edit the OpenAPI specification and additional documentation after you add this asset to a revision.

    If you want to update this information, and the revision is not finalized, you can replace the asset.

    If you want to update this information, and the revision is finalized, you can create a new revision with the updated asset.

  5. Choose Add API stage.

    A job is started to import your asset (in this case, the API) into your data set.

    Note

    If you do not have an API on Amazon API Gateway, you will be prompted to create one.

  6. After the job is finished, the State field in the Jobs section is updated to Completed.

  7. If you have more APIs to add, repeat Step 2.

  8. Under Revision overview, review your revision and its assets.

  9. Choose Finalize.

You have successfully finalized a revision for a data set.

You can edit a revision or delete a revision before you add it to a product.

You are now ready to publish a new API data product.

Adding API assets to a revision (AWS CLI)

You can add API assets by running an IMPORT_ASSET_FROM_API_GATEWAY_API job.

To add API assets to a revision (AWS CLI):
  1. Use the create-job command to add API assets to the revision:

    $ aws dataexchange create-job \ --type IMPORT_ASSET_FROM_API_GATEWAY_API \ --details '{"ImportAssetFromApiGatewayApi":{"DataSetId":"$DATA_SET_ID","RevisionId":"$REVISION_ID","ApiId":"$API_ID","Stage":"$API_STAGE","ProtocolType":"REST"}}' { "Arn": "arn:aws:dataexchange:us-east-1:123456789012:jobs/$JOB_ID", "CreatedAt": "2021-09-11T00:38:19.875000+00:00", "Details": { "ImportAssetFromApiGatewayApi": { "ApiId": "$API_ID", "DataSetId": "$DATA_SET_ID", "ProtocolType": "REST", "RevisionId": "$REVISION_ID", "Stage": "$API_STAGE" } }, "Id": "$JOB_ID", "State": "WAITING", "Type": "IMPORT_ASSET_FROM_API_GATEWAY_API", "UpdatedAt": "2021-09-11T00:38:19.875000+00:00" } $ aws dataexchange start-job --job-id $JOB_ID $ aws dataexchange get-job --job-id $JOB_ID { "Arn": "arn:aws:dataexchange:us-east-1:0123456789012:jobs/$JOB_ID", "CreatedAt": "2021-09-11T00:38:19.875000+00:00", "Details": { "ImportAssetFromApiGatewayApi": { "ApiId": "$API_ID", "DataSetId": "$DATA_SET_ID", "ProtocolType": "REST", "RevisionId": "$REVISION_ID", "Stage": "$API_STAGE" "ApiEndpoint": "string", "ApiKey": "string", "ApiName": "string", "ApiDescription": "string", "ApiSpecificationDownloadUrl": "string", "ApiSpecificationDownloadUrlExpiresAt": "string" } }, "Id": "$JOB_ID", "State": "COMPLETED", "Type": "IMPORT_ASSET_FROM_API_GATEWAY_API", "UpdatedAt": "2021-09-11T00:38:52.538000+00:00" }
  2. Use the list-revision-assets command to confirm that the new asset was created properly:

    $ aws dataexchange list-revision-assets \ --data-set-id $DATA_SET_ID \ --revision-id $REVISION_ID { "Assets": [ { "Arn": "arn:aws:dataexchange:us-east-1:123456789012:data-sets/$DATA_SET_ID/revisions/$REVISION_ID/assets/$ASSET_ID", "AssetDetails": { "ApiGatewayApiAsset": { "ApiEndpoint": "https://$API_ID.execute-api.us-east-1.amazonaws.com/$API_STAGE", "ApiId": "$API_ID", "ProtocolType": "REST", "Stage": "$API_STAGE" } }, "AssetType": "API_GATEWAY_API", "CreatedAt": "2021-09-11T00:38:52.457000+00:00", "DataSetId": "$DATA_SET_ID", "Id": "$ASSET_ID", "Name": "$ASSET_ID/$API_STAGE", "RevisionId": "$REVISION_ID", "UpdatedAt": "2021-09-11T00:38:52.457000+00:00" } ] }

You are now ready to publish the API data product.

Edit a revision
To edit the revision after you’ve finalized it
  1. On the Revision overview, choose De-finalize.

    You see a message that the revision is no longer in the finalized state.

  2. To edit the revision, from Revision overview, choose Actions, Edit.

  3. Make your changes, and then choose Update.

  4. Review your changes and then choose Finalize.

Delete a revision
To delete the revision after you’ve finalized it
  1. On the Revision overview, choose Delete.

  2. Type Delete in the Delete revision dialog box, and then choose Delete.

Warning

This deletes the revision and all of its assets. This action cannot be undone.

Step 5: Publish a new product containing APIs

After you've created at least one data set and finalized a revision with assets, you're ready to publish that data set as a part of a product. For more information, see Product details. Make sure that you have all required details about your product and offer.

You use the AWS Data Exchange console or the AWS Marketplace Catalog API to publish a new product containing APIs. For more information about how to publish a new product using the AWS Marketplace Catalog API, see Using AWS Data Exchange with the AWS Marketplace Catalog API.

Publishing a new product containing APIs (console)
To publish a new product containing APIs
  1. From the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.

  2. From Products, choose Publish new product to open the Publish new product wizard.

  3. In Product visibility:

    1. Choose your product's Product visibility options as either Public or Private.

      All AWS Data Exchange products with visibility set to Public require a public offer.

      For more information, see Product visibility.

    2. Choose your product's Sensitive information configuration.

      For more information, see Sensitive categories of information.

    3. Choose Next.

  4. In Add data:

    1. Under Owned data sets, select the check boxes next to the data sets you want to add, and then choose Add selected.

      Note

      The data sets you choose must have a finalized revision. Data sets without finalized revisions aren't added.

    2. Go to Selected data sets to review your selections.

      You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.

    3. Go to Select revision access rules, and choose the revision access rules that you want to set for data sets included in this product.

      For more information, see Revision access rules.

    4. Choose Next.

  5. In Define product:

    1. Under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

      For more information, see Product details.

    2. (Optional) Under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

      1. In the Edit dialog box, choose Upload to upload a new data dictionary.

        You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.

      2. Choose a saved data dictionary from your computer and then choose Open.

        The data dictionary .csv file appears on the Edit dialog box.

        Note

        Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.

      3. Choose Data dictionary preview to preview the data dictionary.

      4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.

        The samples appear on the Edit dialog box.

        Note

        You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.

      5. Enter a description for each sample that will be visible on the product detail page.

      6. Choose Save.

  6. Under Product definition, enter a Short description and a Long description of your product.

    If you want to use a template for your long description, select Apply template, choose your template type, and then fill out the template with your specific product details.

  7. Choose Next.

  8. Configure your offer in either Add public offer (for public offer) or Add custom offer (for private offers):

    All AWS Data Exchange products with visibility set to Public require a public offer.

    1. For private offers only:

      1. Choose one of the listed Offer types: Private offer, Renewed private offer, or Bring Your Own Subscription (BYOS).

      2. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.

    2. Choose your Pricing and access duration options for the subscription.

    3. For Metered costs - optional, choose Add.

      1. For Add metered cost, select the type of cost for the API call from the Type list:

        • Per API request

        • Per successful API request

        • Per unit of data transferred in bytes

        • New custom metered cost

      2. Enter or update the Cost display name, which is visible on the subscriber’s invoice.

      3. If you're using a Pre-defined metered cost, the Key is automatically generated, can’t be edited, and doesn’t need to be sent back in the response header.

      4. If you're creating a New custom metered cost, enter the Key, which is the identifier for the metered cost in the API response header (15 characters maximum).

        This Key should be sent back as part of the x-amz-dataexchange-metering response header.

        Example Custom key

        If you have a custom key called VertexCount and another custom key called EdgeCount, the “x-amz-dataexchange-metering” response header could have a value of VertexCount=3,EdgeCount=10 or you could return two separate header lines:

        x-amz-dataexchange-metering: VertextCount=3

        x-amz-dataexchange-metering: EdgeCount=10

      5. Enter the price the subscriber is charged per unit in Price / unit.

      6. (Optional) Enter the number of units to display an example of the cost in the Metered cost calculator.

      7. (Optional) Enter a brief Description of the metered cost that appears on the product detail page.

      8. Choose Add.

      9. (Optional) Repeat to add additional metered costs. 

        The order of the metered costs appears on the product detail page. You can’t reorder them.

        Note

        After the offer is created, you can edit the price and description of a metered cost. For more information, see Updating product and offer details.

    4. For private offers only, choose the Offer expiration date by which the subscriber must accept the offer.

    5. Choose your Tax settings, Data subscription agreement (DSA), and Refund policy.

    6. (Optional) For public offers only, set Subscription verification, which enables you to control who can subscribe to this product. For more information, see Subscription verification for providers.

    7. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

    8. Choose Next.

  9. In the Review & publish section, review your product information.

    1. Expand the Product page preview to see how the product page will look after publication.

    2. (Optional) Choose the Edit button in any section to edit that section.

  10. If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product.

On the Product overview page, the status of your product is Awaiting approval and then changes to Published after it's published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom offers associated with the product will not be copied, but public offers will be copied.

To copy a product
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. From the left navigation pane, under Publish data, choose Products.

  3. From Products, choose the button next to the product you want to copy.

  4. Select the Actions dropdown, and then choose Create copy.

  5. Continue through the Publish a new product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing Amazon Redshift data sets

Overview

An Amazon Redshift data set contains AWS Data Exchange datashares for Amazon Redshift. When customers subscribe to a product containing datashares, they are granted read-only access to the tables, views, schemas, and user-defined functions that a data provider adds to the datashare.

As a data provider, you create an AWS Data Exchange for Amazon Redshift datashare in your cluster. Then, you add to the datashare the schemas, tables, views, and user-defined functions that you want the subscribers to access. You then import the datashare to AWS Data Exchange, create a data set, add it to a product, and publish the product. Subscribers are granted access to the datashare upon subscription.

After you have set up your Amazon Redshift datashare in Amazon Redshift, you can create a new Amazon Redshift data set in AWS Data Exchange. You can then create a revision, and add Amazon Redshift datashare assets. This allows requests to the AWS Data Exchange endpoint to proxy through to your Amazon Redshift datashare. You can then add this data set to a product and add pricing. Then, prospective subscribers can view your product and subscribe to it in the AWS Data Exchange catalog.

The following topics describe the process of creating an Amazon Redshift data set and publishing a new product with Amazon Redshift data sets using the AWS Data Exchange console. The process has the following steps:

Step 1: Create an Amazon Redshift datashare asset

Assets are the data in AWS Data Exchange. For more information, see Assets.

To create an Amazon Redshift datashare asset
  1. Create a datashare within your Amazon Redshift cluster.

    For more information about how to create a datashare, see Working with AWS Data Exchange datashares as a producer in the Amazon Redshift Database Developer Guide.

    Note

    We recommend setting your datashare as publicly accessible. If you do not, customers with publicly accessible clusters will not be able to consume your data.

  2. Step 2: Create an Amazon Redshift data set.

Step 2: Create an Amazon Redshift data set

An Amazon Redshift data set includes AWS Data Exchange datashares for Amazon Redshift. For more information, see Amazon Redshift data set.

To create an Amazon Redshift data set
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. On the left side navigation pane, under Publish data, choose Owned data sets.

  3. In Owned data sets, choose Create data set to open the Data set creation steps wizard.

  4. In Select data set type, choose Amazon Redshift datashare.

  5. In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.

  6. Under Add tags – optional, add tags.

  7. Choose Create.

Step 3: Create a revision

In the following procedure, you create a revision after you’ve created a data set in the AWS Data Exchange console. For more information, see Revisions.

To create a revision
  1. On the Data set overview section of the data set details page:

    1. (Optional) Choose Edit name to edit information about your data set.

    2. (Optional) Choose Delete to delete the data set.

  2. On the Revisions section, choose Create revision.

  3. Under Define revision, provide an optional comment for your revision that describes the purpose of the revision.

  4. Under Add tags – optional, add tags associated with the resource.

  5. Choose Create.

  6. Review, edit, or delete your changes from the previous step.

Step 4: Add Amazon Redshift datashare assets to a revision

In the following procedure, you add Amazon Redshift datashare assets to a revision, and then finalize the revision in the AWS Data Exchange console. For more information, see Assets.

To add assets to the revision
  1. Under the AWS Data Exchange datashares for Amazon Redshift section of the data set details page, choose Add datashares.

  2. Under AWS Data Exchange datashares for Amazon Redshift, select the datashares and then choose Add datashare(s).

    Note

    You can add up to 20 datashares to a revision.

    A job is started to import your assets into your revision.

  3. After the job is finished, the State field in the Jobs section is updated to Completed.

  4. If you have more data to add, repeat Step 1.

  5. Under Revision overview, review your revision and its assets.

  6. Choose Finalize.

You have successfully finalized a revision for a data set.

You can edit or delete a revision before you add it to a product.

Step 5: Publish a new product containing Amazon Redshift data sets

After you've created at least one data set and finalized a revision with assets, you're ready to publish a product with Amazon Redshift data sets. For more information, see Product details. Make sure that you have all required details about your product and offer.

To publish a new product containing Amazon Redshift data sets
  1. From the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.

  2. From Products, choose Publish new product to open the Publish new product wizard.

  3. In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.

  4. In the Add data section, under Owned data sets, select the check boxes next to the data sets that you want to add, and then choose Add selected.

    Note

    The data sets you choose must have a finalized revision. Data sets without finalized revisions won't be added.

    1. Go to Selected data sets to review your selections.

      You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.

    2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next.

      For more details, see Revision access rules.

  5. In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

    For more information, see Product details.

  6. (Optional) In the Define product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

    For more information, see Data dictionaries and Samples.

    1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.

      You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.

    2. Choose a saved data dictionary from your computer and then choose Open.

      The data dictionary .csv file appears on the Edit dialog box.

      Note

      Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.

    3. Choose Data dictionary preview to preview it.

    4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.

      The samples appear on the Edit dialog box.

      Note

      You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.

    5. Enter a description for each sample that will be visible on the product detail page.

    6. Choose Save.

  7. Under Product definition, enter a Short description and a Long description of your product.

    If you want to use a template for your long description, select Apply template, choose your template type, and then fill out the template with your specific product details.

  8. Choose Next.

  9. Configure your offer.

    • If you are creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.

      1. Choose your Pricing and access duration options for the subscription.

      2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      3. (Optional) Set Subscription verification, which enables you to control who can subscribe to this product. For more information, see Subscription verification for providers.

      4. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      5. Choose Next.

    • If you are creating a private offer, configure the offer details in the Add custom offer section.

      1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.

      2. Choose your Pricing and access duration options for the subscription.

      3. Choose the Offer expiration date by which the subscriber must accept the offer.

      4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      5. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      6. Choose Next.

  10. In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.

  11. If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval and then changes to Published after it's published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom offers associated with the product will not be copied, but public offers will be copied.

To copy a product
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. From the left navigation pane, under Publish data, choose Products.

  3. From Products, choose the button next to the product you want to copy.

  4. Select the Actions dropdown, and then choose Create copy.

  5. Continue through the Publish a product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing Amazon S3 data access (Preview)

Overview

With AWS Data Exchange for Amazon S3, providers can share direct access to Amazon S3 buckets or specific prefixes and Amazon S3 objects. Providers also use AWS Data Exchange to automatically manage subscriptions, entitlements, billing, and payments.

As a data provider, you can activate direct access to the Amazon S3 bucket locations you manage. Then, you can share access to Amazon S3 objects without creating or managing copies. When a customer subscribes to your data products, AWS Data Exchange automatically provisions an Amazon S3 access point and updates its resource policies on your behalf to grant subscribers read-only access. Subscribers can use the Amazon S3 access point aliases in places where they use Amazon S3 bucket names to access data in Amazon S3.

When the subscription ends, the subscriber’s permissions are revoked. If you choose to end an agreement with a subscriber early, contact AWS Data Exchange support. You can add terms of subscriptions in the Data Subscription Agreement (DSA).

Before you can publish a product containing Amazon S3 data access, you must meet the following prerequisites:

Prerequisites
  • Confirm that the Amazon S3 buckets hosting the data are configured with the Amazon S3 bucket owner enforced setting turned on (ACLs Disabled). For more information, see Controlling ownership of objects and disabling ACLs for your bucket in the Amazon Simple Storage Service User Guide.

  • Confirm that the Amazon S3 buckets hosting the data has encryption disabled or encrypted with Amazon S3 managed keys (SSE-S3). The AWS Data Exchange for Amazon S3 (Preview) delivery method doesn't support AWS KMS keys stored in AWS Key Management Service (AWS KMS) (SSE-AWS KMS) during Preview.

  • Set the Amazon S3 buckets hosting the data to trust AWS Data Exchange owned access points. You must update these Amazon S3 bucket policies to give AWS Data Exchange permissions to create Amazon S3 access points and grant or remove subscribers' access on your behalf.

    An example policy is shown below. Replace <Bucket ARN> with the appropriate value.

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "s3:GetObject", "s3:ListBucket" ], "Resource": [ "<Bucket ARN>", "<Bucket ARN>/*" ], "Condition": { "StringEquals": { "s3:DataAccessPointAccount": [ "337040091392", "504002150500", "366362662752", "330489627928", "291973504423", "461002523379", "036905324694", "540564263739", "675969394711", "108584782536", "844053218156" ] } } } ] }

You can delegate data sharing through AWS Data Exchange to an entire Amazon S3 bucket. However, you can scope delegation to the specific prefixes and objects of the bucket that you want to share in the data set. Following is an example of a scoped policy. Replace <Bucket ARN> and "mybucket/folder1/*" with your own information.

{ "Version": "2012-10-17", "Statement": [ { "Sid": "DelegateToAdxGetObjectsInFolder1", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::mybucket/folder1/*" ], "Condition": { "StringEquals": { "s3:DataAccessPointAccount": [ "337040091392", "504002150500", "366362662752", "330489627928", "291973504423", "461002523379", "036905324694", "540564263739", "675969394711", "108584782536", "844053218156" ] } } }, { "Sid": "DelegateToAdxListObjectsInFolder1", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::mybucket", "Condition": { "StringLike": { "s3:prefix": [ "folder1/*" ] }, "StringEquals": { "s3:DataAccessPointAccount": [ "337040091392", "504002150500", "366362662752", "330489627928", "291973504423", "461002523379", "036905324694", "540564263739", "675969394711", "108584782536", "844053218156" ] } } } ] }

Similarly, to scope access to only a single file, a provider can use the following policy.

{ "Version": "2012-10-17", "Statement": [ { "Sid": "DelegateToAdxGetMyFile", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::mybucket/folder1/myfile" ], "Condition": { "StringEquals": { "s3:DataAccessPointAccount": [ "337040091392", "504002150500", "366362662752", "330489627928", "291973504423", "461002523379", "036905324694", "540564263739", "675969394711", "108584782536", "844053218156" ] } } } ] }

The following topics describe the process of creating an Amazon S3 data set and publishing a new product with Amazon S3 data sets using the AWS Data Exchange console. The process has the following steps:

Step 1: Create an Amazon S3 data set (Preview)

To create an Amazon S3 data set
  1. On the left side navigation pane, under Publish data, choose Owned data sets.

  2. On the left side navigation pane, under Publish data, choose Owned data sets.

  3. In Owned data sets, choose Create data set to open the Data set creation steps wizard.

  4. In Select data set type, choose Amazon S3 data access (Preview)..

  5. In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.

  6. (Optional) Under Add tags – optional, add tags.

  7. Choose Create data set and continue.

Step 2: Configure Amazon S3 data access (Preview)

Choose the Amazon S3 buckets or Amazon S3 bucket locations that you want to make available to subscribers. You can select an entire Amazon S3 bucket, or specify up to five prefixes or objects within an Amazon S3 bucket. To add more Amazon S3 buckets, you must create another Amazon S3 data share.

To configure shared Amazon S3 data access
  1. On the Configure Amazon S3 data access page, select Choose Amazon S3 locations.

  2. In Choose Amazon S3 locations, enter your Amazon S3 bucket name in the search bar or select your Amazon S3 bucket, prefixes, or Amazon S3 files and choose Add selected. Then, choose Add locations.

    Note

    We recommend choosing a top-level folder where a majority of objects and prefixes are stored so providers don't need to reconfigure which prefixes or objects to share.

  3. In Configuration details, choose your Requester Pays configuration. There are two options:

    • Enable Requester Pays (recommended) – Requesters will pay for all requests and transfers in the Amazon S3 bucket. We recommend this option because it helps protect against unintended costs from subscriber requests and transfers.

    • Disable Requester Pays – You pay for subscriber requests and transfers in the Amazon S3 bucket.

      For more information about Requester Pays, see Objects in Requester Pays Buckets in the Amazon Simple Storage Service User Guide.

  4. Choose Save and continue.

Step 3: Review and finalize the data set

Review and finalize your newly created data set. If you wish to create and add another Amazon S3 data access to share access to additional Amazon S3 buckets, prefixes, objects, choose Add another Amazon S3 data access (Preview).

Note

We recommend this when needing to share access to data hosted in a different Amazon S3 bucket than the one previously picked in the initial Amazon S3 data access.

If you would like to make changes prior to publishing, you can save the data set as a draft by choosing Save draft. Then, choose Finalize data set to add it to your product.

Step 4: Add an Amazon S3 data set to an AWS Data Exchange product (Preview)

In the following procedure, you add your data set to a new or existing AWS Data Exchange product.

To add a data set to a new or existing AWS Data Exchange product
  1. On the Owned data sets page, under Data set overview, you can Edit name, Delete, or Create product from data set.

  2. Complete the product creation specifying product description, use cases, metadata, pricing, and terms and conditions.

  3. Review and publish the product when finished.

    Note

    When a customer subscribes to your product, the customer receives access permission to read and use your data using the Amazon S3 access point created on your behalf.

Step 5: Publish a new product containing access to Amazon S3 (Preview)

After you create at least one data set and finalize a revision with assets, you can publish a product with Amazon S3 data access. For more information, see Product details. Make sure that you have all required details about your product and offer.

Note

You don't need to create a new revision when updating the shared Amazon S3 objects unless the Amazon S3 locations have been altered and these objects aren't accessible to subscribers.

To publish a new product containing access to Amazon S3
  1. From the left navigation pane of the AWS Data Exchange console, under Publish data, choose Products.

  2. From Products, choose Publish new product to open the Publish new product wizard.

  3. In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.

  4. In the Add data section, under Owned data sets, select the check boxes next to the data sets that you want to add, and then choose Add selected.

    Note

    The data sets you choose must have a finalized revision. Data sets without finalized revisions aren't added.

    1. Go to Selected data sets to review your selections.

      You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.

    2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next.

      For more details, see Revision access rules.

  5. In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

    For more information, see Product details.

  6. (Optional) In the Define product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

    For more information, see Data dictionaries and Samples.

    1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.

      You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.

    2. Choose a saved data dictionary from your computer and then choose Open.

      The data dictionary .csv file appears on the Edit dialog box.

      Note

      Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.

    3. Choose Data dictionary preview to preview the data dictionary.

    4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.

      The samples appear on the Edit dialog box.

      Note

      You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.

    5. Enter a description for each sample that will be visible on the product detail page.

    6. Choose Save.

  7. Under Product definition, enter a Short description and a Long description of your product.

    If you want to use a template for your long description, select Apply template, choose your template type, and then provide your specific product details in the template.

  8. Choose Next.

  9. Configure your offer.

    • If you're creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.

      1. Choose your Pricing and access duration options for the subscription.

      2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      3. (Optional) Set Subscription verification to control who can subscribe to this product. For more information, see Subscription verification for providers.

      4. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      5. Choose Next.

    • If you're creating a private offer, configure the offer details in the Add custom offer section.

      1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.

      2. Choose your Pricing and access duration options for the subscription.

      3. Choose the Offer expiration date by which the subscriber must accept the offer.

      4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      5. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      6. Choose Next.

  10. In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.

  11. If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval. The status changes to Published after the product is published.

Step 6: (Optional) Copy a product

After you have created your first product, you can copy its details and public offers to create a new product.

Note

You can copy a public, private, published, or unpublished product. Custom offers associated with the product can't be copied, but public offers can be copied.

To copy a product
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. From the left navigation pane, under Publish data, choose Products.

  3. From Products, choose the option next to the product that you want to copy.

  4. Select the Actions dropdown list, and then choose Create copy.

  5. Continue through the Publish a product workflow, with details already filled in, based on the product you chose in Step 3. For more information, see Step 5: Publish a new product.

Publishing a product containing AWS Lake Formation data permission data sets (Preview)

Overview

If you are interested in publishing products containing AWS Lake Formation data permission data sets during this Preview, please email adx-customersuccess@amazon.com.

An AWS Lake Formation data permission data set contains a set of LF-tags and permissions for data managed by AWS Lake Formation. When customers subscribe to a product containing Lake Formation data permissions, they are granted read-only access to the databases, tables, and columns associated with the LF-tags added to the data set.

As a data provider, you start by creating LF-tags in AWS Lake Formation and associating those tags with the data you want to make available to subscribers. For more information about tagging your resources in Lake Formation, see Lake Formation Tag-based access control. Then you import those LF-tags and a set of data permissions into AWS Data Exchange as an asset. Subscribers are granted access to the data associated with those LF-tags upon subscription.

The following topics describe the process of publishing a product containing AWS Lake Formation data permissions. The process has the following steps:

Step 1: Create an AWS Lake Formation data set (Preview)

To create an AWS Lake Formation data set
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. From the left navigation pane, under Publish data, choose Products.

  3. In Owned data sets, choose Create data set to open the Data set creation steps wizard.

  4. In Select data set type, choose AWS Lake Formation data permission.

  5. In Define data set, enter a Name and Description for your data set. For more information, see Data set best practices.

  6. Under Add tags – optional, choose Add new tag.

  7. Choose Create data set and continue.

Step 2: Create an AWS Lake Formation data permission (Preview)

AWS Data Exchange uses LF-Tags to grant data permissions. Choose the LF-Tags that are associated with the data you want to share to grant subscriber permissions to the data.

To create AWS Lake Formation data permission
  1. On the Create Lake Formation data permission page, choose Add LF-Tag.

  2. Enter the Key and choose your LF-Tag Values.

  3. Choose Preview resource(s) to view how your LF-Tags are interpreted.

    1. From Preview resource(s), select your Associated data catalog resource(s).

      Note

      Make sure to revoke IAMAllowedPrincipals group on the following resources. For more information, see Revoking IAM role temporary security credentials.

  4. Review the interpretation of the LF-Tag expression in the dialog box below and Permissions associated with the data set.

  5. For Service access, select your existing service role that allows AWS Data Exchange to assume the role and access, grant, and revoke entitlements to Lake Formation data permissions on your behalf. Then choose Create Lake Formation data permission. For more information about creating a role for an AWS service, see Creating a role to delegate permissions to an AWS service.

  6. In the Define product section, under Product overview, enter information about your product, including the Product name, Product logo, Support contact information, and Product categories.

    For more information, see Product details.

  7. (Optional) In the Define product section, under Data dictionaries and samples – optional, choose a data set by selecting the option button next to the data set name and then choose Edit.

    For more information, see Data dictionaries and Samples.

    1. In the Edit dialog box, under Upload data dictionary, choose Add file to upload a new data dictionary.

      You can choose one data dictionary, in .csv format, with a maximum size of 1 MB.

    2. Choose a saved data dictionary from your computer and then choose Open.

      The data dictionary .csv file appears on the Edit dialog box.

      Note

      Your data dictionary must conform to the AWS Data Exchange data dictionary template. If you don’t have a saved data dictionary to upload, you can choose either the blank data dictionary template link or the example data dictionary link in the AWS Data Exchange console.

    3. Choose Data dictionary preview to preview the data dictionary.

    4. Under Samples - optional, choose Upload samples, choose a sample from your computer, and then choose Open.

      The samples appear on the Edit dialog box.

      Note

      You can upload up to 10 samples with a maximum size of 50 MB. Samples in .csv format can be previewed.

    5. Enter a description for each sample that will be visible on the product detail page.

    6. Choose Save.

  8. Under Product definition, enter a Short description and a Long description of your product.

    If you want to use a template for your long description, select Apply template, choose your template type, and then provide your specific product details in the template.

  9. Choose Next.

  10. Configure your offer.

    • If you're creating a public offer, in the Add public offer section, configure your offer. All AWS Data Exchange products with visibility set to Public require a public offer.

      1. Choose your Pricing and access duration options for the subscription.

      2. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      3. (Optional) Set Subscription verification to control who can subscribe to this product. For more information, see Subscription verification for providers.

      4. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      5. Choose Next.

    • If you're creating a private offer, configure the offer details in the Add custom offer section.

      1. In the Subscriber account information section, add at least one subscriber account to which you want to extend the offer.

      2. Choose your Pricing and access duration options for the subscription.

      3. Choose the Offer expiration date by which the subscriber must accept the offer.

      4. Choose your US sales tax settings, data subscription agreement (DSA), and refund policy.

      5. Choose your Offer auto-renewal option. For more information, see Creating an offer for AWS Data Exchange products.

      6. Choose Next.

  11. In the Review & publish section, review your product information and then expand the Product page preview to see how it will look after it’s published.

  12. If you're sure that you want to make the product and public offer visible and available to everyone, choose Publish.

You've now completed the manual portion of publishing a data product with a public offer. AWS Data Exchange prepares and publishes your product. On the Product overview page, the status of your product is Awaiting approval. The status changes to Published after the product is published.

Step 3: Review and finalize

After creating your AWS Lake Formation data permission (Preview), you can Review and finalize your data set.

To review and finalize
  1. Review your Data set details and Tags in Step 1 for accuracy.

  2. Review your LF-Tag expression(s), Add another Lake Formation data permission (optional), Associated data catalog resource(s), and job details.

    Note

    Job are deleted 90 days after they’re created.

  3. Choose Finalize.

Step 5: (Optional) Create a revision

To create a revision
  1. From the Owned data sets section, choose the data set for which you want to add a revision.

  2. Choose the Revisions tab.

  3. In the Revisions section, choose Create revision.

  4. On the Revise Lake Formation data permission page, choose Add LF-Tag.

  5. Review the Permissions for Database and Table.

  6. From Service access, select an existing service role and then choose Create Lake Formation data permission.

Step 6: Publish a new product containing AWS Lake Formation data sets (Preview)

After you've created at least one data set and finalized a revision with assets, you're ready to publish a product with AWS Lake Formation data sets. For more information, see Product details. Make sure that you have all required details about your product.

To publish a new product containing AWS Lake Formation data sets (Preview)
  1. Open your web browser and sign in to the AWS Data Exchange console.

  2. From the left navigation pane, under Publish data, choose Products.

  3. From Products, choose Publish new product to open the Publish new product wizard.

  4. In the Product visibility section, choose your product's Product visibility options and Sensitive information configuration, and then choose Next. For more information, see Product visibility and Sensitive categories of information.

  5. In the Add data section, under Owned data sets, select the check boxes next to the data sets that you want to add, and then choose Add selected.

    Note

    The data sets you choose must have a finalized revision. Data sets without finalized revisions aren't added.

    1. Go to Selected data sets to review your selections.

      You can review the Name of the data set, the Type of data set, and the timestamp of when the data set was Last updated.

    2. Go to Select revision access rules, choose the revision access rules that you want to set for data sets included in this product, and then choose Next.

      For more details, see Revision access rules.

Considerations when publishing an AWS Lake Formation data permission data set (Preview)

To ensure an optimal subscriber experience, we strongly advise against making any of the following modifications to any permissions where your product contains AWS Data Exchange for Lake Formation data sets (Preview) with active subscribers to that product.

  • We recommend not deleting or modifying IAM roles passed to AWS Data Exchange in published products containing AWS Lake Formation data sets. If you delete or modify such IAM roles, the following issues occur:

    • AWS accounts that have access to the Lake Formation data permissions might retain access indefinitely.

    • AWS accounts that subscribe to your product but have not yet received access to the Lake Formation data permissions will fail to receive access.

    AWS Data Exchange will not be liable for any IAM roles that you delete or modify.

  • We recommend that you don’t revoke granted AWS Lake Formation data permissions from IAM roles passed to AWS Data Exchange in published product containing AWS Lake Formation data sets. If you revoke granted data permissions from such IAM roles, the following issues occur:

    • AWS accounts that have access to the Lake Formation data permissions might retain access indefinitely.

    • AWS accounts that subscribe to your product but have not yet received access to the Lake Formation data permissions will fail to receive access.

  • We recommend not revoking granted AWS Lake Formation data permissions from AWS accounts with active subscriptions to published products containing AWS Lake Formation data sets. If you revoke granted data permissions from AWS accounts subscribed to your product, those accounts will lose access, creating a poor customer experience.

  • We recommend setting the cross account version in your AWS Glue Data Catalog to version 3 when publishing products containing AWS Lake Formation data sets. If you downgrade the cross account version of your Data Lake Catalog while having published products containing AWS Lake Formation data sets, the AWS accounts that subscribe to your product but have not yet received access to the Lake Formation data permissions may fail to get access to the data.