Menu
AWS Data Pipeline
Developer Guide (API Version 2012-10-29)

Step 1: Create the Pipeline

First, create the pipeline.

To create the pipeline

  1. Open the AWS Data Pipeline console at https://console.aws.amazon.com/datapipeline/.

  2. The first screen that you see depends on whether you've created a pipeline in the current region.

    1. If you haven't created a pipeline in this region, the console displays an introductory screen. Choose Get started now.

    2. If you've already created a pipeline in this region, the console displays a page that lists your pipelines for the region. Choose Create new pipeline.

  3. In Name, enter a name for your pipeline.

  4. (Optional) In Description, enter a description for your pipeline.

  5. For Source, select Build using a template, and then select the following template: Import DynamoDB backup data from S3.

  6. Under Parameters, set Input S3 folder to s3://elasticmapreduce/samples/Store/ProductCatalog, which is a sample data source, and set DynamoDB table name to the name of your table.

  7. Under Schedule, choose on pipeline activation.

  8. Under Pipeline Configuration, leave logging enabled. Choose the folder icon under S3 location for logs, select one of your buckets or folders, and then choose Select.

    If you prefer, you can disable logging instead.

  9. Under Security/Access, leave IAM roles set to Default.

  10. Click Edit in Architect.

Next, configure the Amazon SNS notification actions that AWS Data Pipeline performs depending on the outcome of the activity.

To configure the success and failure actions

  1. In the right pane, click Activities.

  2. From Add an optional field, select On Success.

  3. From the newly added On Success, select Create new: Action.

  4. From Add an optional field, select On Fail.

  5. From the newly added On Fail, select Create new: Action.

  6. In the right pane, click Others.

  7. For DefaultAction1, do the following:

    1. Change the name to SuccessSnsAlarm.

    2. From Type, select SnsAlarm.

    3. In Topic Arn, enter the ARN of the topic that you created (see ARN resource names for Amazon SNS.

    4. Enter a subject and a message.

  8. For DefaultAction2, do the following:

    1. Change the name to FailureSnsAlarm.

    2. From Type, select SnsAlarm.

    3. In Topic Arn, enter the ARN of the topic that you created (see ARN resource names for Amazon SNS.

    4. Enter a subject and a message.