Amazon Mobile Analytics
User Guide

Migrating from Auto Export to Amazon Pinpoint Event Streams

Starting April 30, 2018, Amazon Mobile Analytics no longer supports auto export, a feature that exports event data from your apps to an Amazon S3 bucket or an Amazon Redshift cluster. Auto export is replaced by Amazon Pinpoint event streams. With event streams, you can stream your event data in real time to Amazon Kinesis Data Streams or Amazon Kinesis Data Firehose.

If you stream to Kinesis Data Firehose, you can use a delivery stream to send your data to AWS data stores, including Amazon Simple Storage Service (Amazon S3) or Amazon Redshift. You can replace auto export with event streams by streaming your event data to the same Amazon S3 bucket or Amazon Redshift cluster that receives your event data with auto export.

If your app reports events to Mobile Analytics using the AWS Mobile SDK, you can stream that data by using Amazon Pinpoint.

For more information about event streams, see Streaming App and Campaign Events with Amazon Pinpoint in the Amazon Pinpoint User Guide.

Complete the following steps to migrate from using auto export to using event streams. To migrate, first, you create a Kinesis Data Firehose delivery stream. Then, you enable event streaming in Amazon Pinpoint. Finally, when you're ready to stop using auto export, you disable it.

This migration doesn't require you to update your app, but we recommend that you update your app to integrate Amazon Pinpoint by using the AWS Mobile SDK.

Step 1: Create a Kinesis Data Firehose Delivery Stream

Create a delivery stream that sends data to the same Amazon S3 bucket or Amazon Redshift cluster that receives your data from auto export.

Option A: Create a Delivery Stream with an Amazon S3 Destination

Complete the following steps if you use auto export to send your event data to an Amazon S3 bucket.

To create the delivery stream

  1. Sign in to the AWS Management Console, and open the Amazon Mobile Analytics console at https://console.aws.amazon.com/mobileanalytics/home.

  2. Choose Export Your Data.

  3. Under Auto Export Settings, note the value that's provided in the Amazon S3 Bucket column. You will need this bucket name when you create your delivery stream.

  4. Open the Kinesis Data Firehose console at https://console.aws.amazon.com/firehose/.

  5. Choose Create delivery stream.

  6. In Step 1: Name and source, specify a Delivery stream name.

  7. Choose Next.

  8. In Step 2: Transform record, choose Next.

  9. In Step 3: Choose destination, for Destination, choose Amazon S3.

  10. Under S3 destination, for S3 bucket, choose the bucket name that you obtained from the Mobile Analytics console.

  11. (Optional) For Prefix, specify a prefix for the event data that's streamed by Amazon Pinpoint, such as pinpoint.

  12. Choose Next.

  13. (Optional) In Step 4: Configure settings, customize the buffer, compression, encryption, and error logging settings as needed. For more information, see Configure settings in the Amazon Kinesis Data Firehose Developer Guide.

  14. For IAM role, choose Create new, or Choose.

  15. On the role creation page, for IAM Role, choose Create a new IAM Role. You can customize Role Name to make the role easier to recognize. Then, choose Allow.

  16. Choose Next.

  17. In Step 5: Review, verify that your delivery stream settings are correct, and choose Create delivery stream.

Option B: Create a Delivery Stream with an Amazon Redshift Destination

Complete the following steps if you use auto export to send your event data to an Amazon Redshift cluster. First, you find the resources in your AWS account that are used by auto export. You provide this information when you create a delivery stream. Then, you update your cluster with a new table that receives the event data. Finally, you create the delivery stream.

To find the auto export resources in your AWS account

  1. Sign in to the AWS Management Console, and open the Amazon Mobile Analytics console at https://console.aws.amazon.com/mobileanalytics/home.

  2. Choose Export Your Data.

  3. Under Auto Export Settings, note the value that's provided in the Amazon S3 Bucket column. You need this bucket name when you create your delivery stream.

  4. Under Auto Export Settings, choose the Click Here to Open Stack in the CloudFormation Console link. The AWS CloudFormation console opens, and the stack that includes your auto export resources is selected.

  5. Choose the Outputs tab.

  6. Note the value for the RedshiftClusterId key. You need this ID when you create your delivery stream.

To create an Amazon Redshift database table for your event data

  1. Using a SQL query tool (for example, SQL Workbench/J), connect to your Amazon Redshift cluster. For more information, see Connecting to Clusters from Client Tools and Code in the Amazon Redshift Cluster Management Guide.

    Note

    To connect to your cluster, you require the master user password. You provided this password to Mobile Analytics when you originally enabled auto export.

  2. Use the CREATE TABLE SQL command to create a table in the events database. In your command, use the LIKE parameter to copy the AWSMA.event table, which is the table that Mobile Analytics writes your event data to. Your new table will hold the event data that you stream with Amazon Pinpoint. Note the table name because you need it when you create your delivery stream.

  3. Use the CREATE USER SQL command to create a database user account. Kinesis Data Firehose uses this account to connect to your cluster and add event data to the table you created. Note the user name and password because you provide these values when you create your delivery stream.

    Note

    You must grant the user INSERT permissions so that the delivery stream can add data to your table.

To create the delivery stream

  1. Open the Kinesis Data Firehose console at https://console.aws.amazon.com/firehose/.

  2. Choose Create delivery stream.

  3. In Step 1: Name and source, specify a name for Delivery stream name.

  4. Choose Next.

  5. In Step 2: Transform record, choose Next.

  6. In Step 3: Choose destination, for Destination, choose Amazon Redshift.

  7. Under Amazon Redshift destination, for Cluster, choose the cluster ID that you obtained from the AWS CloudFormation console.

  8. For User name and Password, provide the user name and password for the database user account that you created.

  9. For Database, type events.

  10. For Table, specify the table that you created.

  11. Under Intermediate S3 destination, for Intermediate S3 bucket, choose the name of the Amazon S3 bucket that you obtained from the Mobile Analytics console.

  12. (Optional) For Prefix, specify a prefix for the event data that's streamed by Amazon Pinpoint, such as pinpoint.

  13. Under Amazon Redshift COPY command, for COPY options, enter the following:

    JSON 's3://s3-bucket/jsonpaths/awseventexportjsonpaths.json' TRUNCATECOLUMNS TIMEFORMAT 'epochmillisecs'

    For s3-bucket, provide the bucket name that you obtained in the Mobile Analytics console.

  14. Choose Next.

  15. (Optional) In Step 4: Configure settings, customize the buffer, compression, encryption, and error logging settings as needed. For more information, see Configure settings in the Amazon Kinesis Data Firehose Developer Guide.

  16. For IAM role, choose Create new, or Choose.

  17. On the role creation page, for IAM Role, choose Create a new IAM Role. You can customize the Role Name to make the role easier to recognize. Then, choose Allow.

  18. Choose Next.

  19. On Step 5: Review, verify that your delivery stream settings are correct, and choose Create delivery stream.

Step 2: Enable Event Streaming with Amazon Pinpoint

Amazon Pinpoint receives the event data that's reported by your app. Enable event streaming in Amazon Pinpoint, and send your data to the delivery stream that you created.

  1. Open the Amazon Pinpoint console at https://console.aws.amazon.com/pinpoint/.

  2. The console opens on the Projects page, which lists your Amazon Pinpoint projects. These projects include the apps that you defined in Mobile Analytics. Choose the app that you're migrating from auto export to event streaming.

  3. In the navigation pane on the left, choose Settings.

  4. On the Settings page, choose Event stream.

  5. Choose Enable streaming of events to Amazon Kinesis.

  6. Under Stream to Amazon Kinesis, choose Send events to an Amazon Kinesis delivery stream.

  7. For Amazon Kinesis Firehose delivery stream, choose the delivery stream that you created.

  8. For IAM role, choose Automatically create a role, and specify a name for the role.

  9. Choose Save. Your event stream is enabled. As Amazon Pinpoint receives events from your app, it sends the event data to your delivery stream.

Step 3: Disable Auto Export with Mobile Analytics

After you verify that Amazon Pinpoint is streaming the data that you expect, and after you're ready to stop using auto export, disable it.

  1. Sign in to the AWS Management Console, and open the Amazon Mobile Analytics console at https://console.aws.amazon.com/mobileanalytics/home.

  2. Choose Export Your Data.

  3. Under Manage Apps, choose the app that you're disabling auto export for.

  4. Choose Configure Auto Export to S3 or Configure Auto Export to Redshift + S3 (depending on how you configured auto export), and choose Disable Auto Export.

    After you've confirmed the action, data for the chosen apps stops flowing to your S3 bucket and Amazon Redshift cluster. Disabling auto export doesn't delete your S3 bucket, the data contained in it, your Amazon Redshift cluster, or your Amazon EC2 instance.