Creating Pipelines Using Console Templates - AWS Data Pipeline

Creating Pipelines Using Console Templates

The AWS Data Pipeline console provides several pre-configured pipeline definitions, known as templates. You can use templates to get started with AWS Data Pipeline quickly. You can also create templates with parametrized values. This allows you to specify pipeline objects with parameters and pre-defined attributes. You can then use a tool to create values for a specific purpose within the pipeline. This allows you to reuse pipeline definitions with different values. For more information, see Creating a Pipeline Using Parametrized Templates.

Initialize, Create, and Schedule a Pipeline

The AWS Data Pipeline console Create Pipeline page allows you to create and schedule a pipeline easily.

To create and schedule a pipeline

  1. Open the AWS Data Pipeline console at

  2. Click either Get started now or Create Pipeline.

  3. Enter a pipeline name and an optional description for the pipeline.

  4. Choose Build using Architect to interactively create and edit nodes in a pipeline definition or Build using a template to select a template. For more information about templates, see Choose a Template.

    If you use choose to use a template, the console displays a form that is specific to that template under Parameters. Complete the form as appropriate.

  5. Choose whether to run the pipeline once on activation or on a schedule.

    If you choose to run the pipeline on a schedule:

    1. For Run every, choose a period for the pipeline. The start and end time must define an interval that's long enough to accommodate this period.

    2. Choose a Starting time. If you choose on pipeline activation, the pipeline uses the current activation time.

    3. Choose an Ending time. If you choose never, the pipeline runs indefinitely.

  6. Select an option for IAM Roles. If you select Default, AWS Data Pipeline assigns its own default roles. You can optionally select Custom to choose other roles available to your account.

  7. Choose either Edit in Architect or Activate.

Choose a Template

When you choose a template, the pipeline create page populates with the parameters specified in the pipeline definition, such as custom Amazon S3 directory paths, Amazon EC2 key pair names, database connection strings, and so on. You can provide this information at pipeline creation and activation. The following templates available in the console are also available for download from the Amazon S3 bucket: s3://datapipeline-us-east-1/templates/.