Getting started with Amazon AppFlow - Amazon AppFlow

Getting started with Amazon AppFlow

This tutorial provides a hands-on introduction to Amazon AppFlow. You create and configure a flow to move data between a data source and a data destination.

Prerequisites

  • If you do not have an AWS account, you must create one. For more information, see How to create and activate a new AWS account.

  • Verify that you have the required information about the source and destination and that they meet any configuration requirements. For more information, see Requirements for Amazon AppFlow.

  • Amazon AppFlow encrypts your data and connection details during transit and at rest. For more information, see Data protection in Amazon AppFlow. When you configure a flow, you specify an AWS Key Management Service CMK to use for encryption. You can choose the AWS managed customer master key (CMK) that Amazon AppFlow creates by default, named AWSDefaultEncryptionKey or you can choose a customer managed CMK that you create. To create a CMK, see Creating symmetric CMKs in the AWS Key Management Service Developer Guide.

  • If you access AWS as an IAM user, your administrator must grant you the permissions required to create and run flows. For more information, see Identity and access management for Amazon AppFlow.

Step 1: Create a flow

Provide basic information for your flow.

To create a flow

  1. Open the Amazon AppFlow console at https://console.aws.amazon.com/appflow/.

  2. Choose Create flow.

  3. For Flow details, enter a name and description for the flow.

  4. (Optional) To use a customer managed CMK instead of the default AWS managed CMK, choose Data encryption, Customize encryption settings and then select an existing CMK or create a new one.

  5. (Optional) To add a tag, choose Tags, Add tag and then enter the key name and value.

  6. Choose Next.

Step 2: Configure the flow

Provide information about the source and destination for your flow.

To configure the flow

  1. For Source details, select the source and provide the requested information. For example, provide connection information and select objects or events.

  2. For Destination details, select the destination and provide the requested information about the location.

  3. For Flow trigger, choose how to trigger the flow. The following are the flow trigger options:

    • Run on demand - Activate the flow manually.

    • Run on event - Activate the flow based on the specified change event. This option is available only for SaaS applications that provide change events. You must choose the event when you choose the source.

    • Run on schedule - Activate the flow on the specified schedule and transfer the specified data. The scheduling frequency depends on the frequency supported by the source application.

  4. Choose Next.

Tip

Attempting a connection with an expired user login can return a 'status code 400' error. If you encounter this error, we recommend creating a new connection and deleting the old one, or using an existing connection with valid credentials.

Step 3: Map data fields

Map the fields in the source objects to fields in the destination. This determines how data is transferred from the source to the destination.

To map data fields

  1. For Mapping method, choose how to map the fields and complete the field mapping. The following are the field mapping options:

    • Manually map fields - Use the Amazon AppFlow user interface to specify the field mapping. To map all fields, choose Source field name, Bulk actions, Map all fields directly. Otherwise, select one or more fields from Source field name, Source fields, and then choose Map fields directly.

    • Upload a .csv file with mapped fields - Use a comma-separated values (CSV) file to specify the field mappings. Each line in the CSV file contains the source field name, followed by a comma, which is followed by the destination field name. For more information on how to create the CSV file for upload, see the note that follows this procedure.

  2. (Optional) To add a formula that concatenates fields, select two fields from Mapped fields and then choose Add formula.

  3. (Optional) To mask or truncate field values, select one or more fields from Mapped fields and then choose Modify values.

  4. (Optional) For Validations, add validations to check whether a field has bad data. For each field, choose the condition that indicates bad data and what action Amazon AppFlow should take when a field in a record is bad.

  5. Choose Next.

Tip

When manually mapping between a source and destination, you must select compatible fields and be sure not to exceed the number of records supported by the destination. For more information on supported record quotas, see Quotas for Amazon AppFlow in the Amazon AppFlow User Guide.

Note

When creating a CSV file to upload to Amazon AppFlow, you must specify each source field and destination field pair in a single line separated by a comma. For example, if you want to map source fields SF1, SF2, and SF3 to destination fields DFa, DFb, and DFc respectively, the CSV file should contain three lines as follows:

SF1, DFa

SF2, DFb

SF3, DFc

Save your file with a .csv extension and then upload this file to import the mapping into Amazon AppFlow.

Step 4 (Optional): Add filters

Specify a filter to determine which records to transfer. Amazon AppFlow enables you to filter data fields by adding multiple filters and by adding criteria to a filter.

Note

When you select field names with string values, OR logic allows you to combine two or more criteria into a broader condition. When you add multiple filters, AND logic allows you to combine your filters into a narrower condition.

To add filters

  1. To add a filter, choose Add filter, select the field name, select a condition, and then specify the criteria.

  2. (Optional) To add further criteria to your filter, choose Add criteria. Depending on the field and the condition, you can add up to 10 criteria per filter.

  3. (Optional) To add another filter, choose Add filter again. You can create up to 10 filters to specify which data fields you want to use in your flow. Amazon AppFlow will implement each filter in the order in which you specify them, and transfer only the records that meet all filter criteria.

  4. To remove a filter, choose Remove next to the filter.

  5. When you are finished adding filters, choose Next.

Step 5: Review and create

Review the information for your flow. To change the information for a step, choose Edit. When you are finished, choose Create flow.

Tip

If the flow creation fails, review the error message and confirm that all required fields have been entered.