Menu
Amazon Simple Email Service
Developer Guide (API Version 2010-12-01)

Step 4: Create an Amazon Kinesis Analytics Application

Now that you have set up event publishing with Amazon SES, you can configure Amazon Kinesis Analytics to capture the email sending event data from your Amazon Kinesis Firehose delivery stream. To do this, you create an Amazon Kinesis Analytics application.

The following procedure shows how to use the Amazon Kinesis Analytics console to create an application that captures Amazon SES email sending event data from your Firehose delivery stream, and then how to perform a simply SQL query on the data to return the events of type "Send".

Note

The email sending events of different event types (send, bounce, complaint, and delivery) have different JSON schemas. In a production environment, you might examine several fields of this schema, but in this tutorial, we limit our examination to a small set of fields that are present for all event types.

To create an Amazon Kinesis Analytics application

  1. Start sending a steady stream of emails configured for event publishing through Amazon SES, and continue sending the emails throughout this procedure. This is required so that Amazon Kinesis Analytics can automatically detect the schema of the event records. Sending one email every ten seconds for five minutes is sufficient for this tutorial. For more information, see Step 3: Send Emails.

    After your email program has sent a few emails, move to the next step.

  2. Sign in to the AWS Management Console and open the Analytics console at https://console.aws.amazon.com/kinesisanalytics.

  3. Choose Create new application.

  4. Enter an application name and description, and then choose Save and continue.

  5. Choose Connect to a source.

  6. Choose the Firehose stream you created in Step 2: Set up a Configuration Set.

    Amazon Kinesis Analytics attempts to discover the schema of the email sending event records based on the incoming records. If Amazon Kinesis Analytics displays Error discovering input schema, that means that Amazon Kinesis Analytics has not received any email sending records yet. Choose Rediscover schema. You might need to choose this button several times. If schema discovery does not succeed after several attempts, ensure that your email sending application is steadily sending emails, and that the emails specify a configuration set.

    When Amazon Kinesis Analytics detects a schema, it displays a success message and lists the records it detected.

    Important

    Do not choose Save and continue. This will cause errors because the discovered schema does not adhere to SQL naming constraints. You must edit the schema as described in the next step.

  7. Choose Edit schema.

  8. For this tutorial, we remove most of the rows. Choose X next to all rows except rows with the following column names:

    • eventType

    • timestamp

    • messageId

    • to

    • ses:configuration-set

    Important

    Do not choose Save schema and update stream samples. This will cause errors because the discovered schema does not adhere to SQL naming constraints. You must edit the schema as described in the next step.

  9. Examine the remaining entries under Column name and compare them to the SQL naming requirements as follows:

    • Format – As described in Identifiers in the Amazon Kinesis Analytics SQL Reference, unquoted identifiers must start with a letter or underscore, and be followed by letters, digits, or underscores. Amazon SES auto-tag names do not comply with these requirements because they contain colons and dashes. You will edit these in the next step.

    • Reserved words – Column names must not conflict with the SQL reserved words listed in Reserved Words and Keywords in the Amazon Kinesis Analytics SQL Reference. Examples of reserved keywords that conflict with Amazon SES event records are timestamp, value, date, from, and to.

  10. Edit the remaining column names to conform to the SQL requirements as follows:

    • Rename ses:configuration-set to ses_configuration_set.

    • Rename timestamp to ses_timestamp.

    • Rename to to ses_to.

  11. Choose Save schema and update stream samples. If you encounter validation errors, ensure that you correctly performed step 10. If you encounter the No rows in source stream error, ensure that you are still sending the email stream that you started at the beginning of this procedure, and then choose Retrieve rows. You might need to choose Retrieve rows several times before Amazon Kinesis Analytics captures records.

  12. Upon successful retrieval of rows, choose Exit (done).

Next Step

Step 5: Run a SQL Query

On this page: