Automated Deployment - Amazon Personalize Optimizer Using Amazon Pinpoint Events

Automated Deployment

Before you launch the automated deployment, please review the architecture, configuration, storage security, and other considerations discussed in this guide. Follow the step-by-step instructions in this section to configure and deploy the solution into your account.

Time to deploy: Approximately 3 minutes

Prerequisites

Verify that you have the following setups before deploying the solution:

  • A configured and deployed Amazon Personalize campaign

  • A configured Amazon Pinpoint project

  • A recommender model in Amazon Pinpoint

Amazon Personalize Campaign

A fully deployed Amazon Personalize campaign with the following interaction schema is required.

{ "type": "record", "name": "Interactions", "namespace": "com.amazonaws.personalize.schema", "fields": [ { "name": "USER_ID", "type": "string" }, { "name": "ITEM_ID", "type": "string" }, { "name": "EVENT_TYPE", "type": "string" }, { "name": "EVENT_VALUE", "type": "float" }, { "name": "TIMESTAMP", "type": "long" } ], "version": "1.0" }

If your interaction schema is different, you must update the Athena query so that it generates a CSV file that matches your schema. For instructions to edit the Athena query, see Appendix C.

Amazon Pinpoint Project

An Amazon Pinpoint project in the same AWS Region where you plan to deploy this solution must be configured. If one does not already exist, take the following steps to create one.

  1. Navigate to the Amazon Pinpoint console.

  2. In the All projects section, choose Create a project.

  3. In the Create a project dialog box, enter a Project name and choose Create.

  4. On the Configure features page, review the URL in the address bar and record the project identifier, which is the hexadecimal string between /apps/ and /setup. For example, 1xxxx8xx055x40x89xxx77830xxx9x27.

    Note

    The project identifier is needed when you launch the solution. Enter the project identifier as the value for the Amazon Pinpoint Project ID parameter. If you've used the Amazon Pinpoint API, you may have seen references to applications. In Amazon Pinpoint, a project is the same as an application. This solution uses the term Project ID instead of Application ID.

The project is automatically saved.

Recommender Model in Amazon Pinpoint

Verify that you have a recommender model available in Amazon Pinpoint. Take the following steps if you need to create one.

  1. Navigate to the Amazon Pinpoint console.

  2. Choose your Amazon Pinpoint project and then, from the left navigation pane, select Machine learning models.

  3. Choose Add recommender model and set up the model following the instructions in Setting Up a Recommender Model in Amazon Pinpoint in the Amazon Pinpoint User Guide.

Note

Update the AWS Lambda function that runs the processing method to emit custom events to Amazon Pinpoint. This Lambda function tracks the recommended items to end users using the Amazon Pinpoint campaign. See Step 2 for the instructions to make this update.

What We'll Cover

The procedure for deploying this architecture on AWS consists of the following steps. For detailed instructions, follow the links for each step.

Step 1. Launch the Stack

  • Launch the AWS CloudFormation template into your AWS account.

  • Enter values for required parameters: Stack Name, Amazon Personalize Campaign ARN, Amazon Personalize Solution ARN, Amazon Personalize Interaction Dataset ARN, and Amazon Pinpoint Project ID.

  • Review the other template parameters and adjust, if necessary.

Step 2. Update the Processing Method AWS Lambda Function Used in Amazon Pinpoint Recommender Model Configuration

  • Update the AWS Lambda function that is currently configured in your Amazon Pinpoint recommender model configuration.

Step 3. Verify Updated Amazon Personalize Campaign

  • Verify that a new Amazon Personalize solution version is created and the campaign is updated.

Step 1. Launch the Stack

This automated AWS CloudFormation template deploys the Amazon Personalize Optimizer Using Amazon Pinpoint Events solution in the AWS Cloud.

Note

You are responsible for the cost of the AWS services used while running this solution. See the Cost section for more details. For full details, see the pricing webpage for each AWS service you will be using in this solution.

  1. Sign in to the AWS Management Console and click the button below to launch the AWS CloudFormation template.

    
                                Amazon Personalize Optimizer Using Amazon Pinpoint Events launch button

    You can also download the template as a starting point for your own implementation.

  2. The template is launched in the US East (N. Virginia) Region by default. To launch this solution in a different AWS Region, use the Region selector in the console navigation bar.

    Note

    This solution uses the Amazon Pinpoint and Amazon Personalize services, which are currently available in specific AWS Regions only. Therefore, you must launch this solution in an AWS Region where Amazon Pinpoint and Amazon Personalize are both available. For the most current availability by Region, see AWS service offerings by Region.

  3. On the Create stack page, verify that the correct template URL shows in the Amazon S3 URL text box and choose Next.

  4. On the Specify stack details page, assign a name to your solution stack.

  5. Under Parameters, review the parameters for the template and modify them as necessary. This solution uses the following parameters.

    Parameter Default Description
    Amazon Personalize Campaign ARN <Requires input>

    The full ARN from the Amazon Personalize campaign that is currently configured in your account.

    Amazon Personalize Solution ARN <Requires input>

    The full ARN from the Amazon Personalize solution that is used by the campaign.

    Amazon Personalize Interaction Dataset ARN <Requires input>

    The full ARN from the Amazon Personalize interaction dataset used by the campaign.

    Pinpoint Project ID <Requires input>

    An Amazon Pinpoint Project ID that has the Amazon Personalize recommender configured.

    Pinpoint Event Types _campaign.opened_notification, _email.open, _email.click, _email.unsubscribe

    Name of the events emitted by the Amazon Pinpoint Event Stream that will be the EVENT_TYPE values when the interaction dataset is generated. For information about updating this parameter post-deployment, see Appendix B.

    Pinpoint Event Type Interaction Values 100, 50, 100, -200

    Event values that correspond to the Amazon Pinpoint Event Types that will be the EVENT_VALUE values when the interaction dataset is generated. For information about updating this parameter post-deployment, see Appendix B.

    Interaction History Date Scope -1

    Number of days to look back in the query to build out the new interaction dataset. A value of -1 will query all historical event data in Amazon S3.

  6. Choose Next.

  7. On the Configure stack options page, choose Next.

  8. On the Review page, review and confirm the settings. Be sure to check the box acknowledging that the template will create AWS Identity and Access Management (IAM) resources.

  9. Choose Create stack to deploy the stack.

    You can view the status of the stack in the AWS CloudFormation console in the Status column. You should see a status of CREATE_COMPLETE in approximately 3 minutes.

    Note

    In addition to the series of AWS Lambda functions used to query data in Amazon S3, an additional Lambda function, CustomResourceHelper, runs only during initial configuration or when resources are updated or deleted.

    When running this solution, you will see all the Lambda functions in the AWS Management Console, but only the ones used to query data in Amazon S3 are regularly active. Do not delete the CustomResourceHelper function as it is necessary to manage associated resources.

Step 2. Update the Processing Method AWS Lambda Function Used in Amazon Pinpoint Recommender Model Configuration

Update the AWS Lambda function that is used in the Amazon Pinpoint recommender model configuration to allow the solution to track the item(s) that were recommended by Amazon Personalize when querying for new interaction data. Alter the AWS Lambda function to emit a custom event to Amazon Pinpoint for tracking purposes and to add permission to the role that runs the Lambda function.

  1. Navigate to the AWS Lambda console.

  2. In the navigation pane, choose Functions.

  3. Select the AWS Lambda function that is currently configured in the Amazon Pinpoint recommender model configuration settings as the processing method for your campaign.

  4. In the Function code section, edit the existing code using the examples below to call the Amazon Pinpoint Event API to emit a custom event with EventType of _custom.recommender.

    Note

    Configure the Amazon Pinpoint event in the sample below to emit the correct personalize_user_id value that is being used in your Amazon Personalize integration with Amazon Pinpoint. The integration allows for either the EndpointID or the UserID. For additional information about Amazon Pinpoint recommender model configuration settings, see Setting Up a Recommender Model in Amazon Pinpoint in the Amazon Pinpoint User Guide.

    The following code is an example NodeJS implementation

    const AWS = require(‘aws-sdk’); const pinpoint = new AWS.Pinpoint(); exports.handler = async (event) => { await emitRecommenderEvents(event); //… normal processing ... return event.Endpoints; } const emitRecommenderEvents = async function(event) { const batchEventItems = {}; Object.keys(event.Endpoints).forEach((endpointId, ind) => { batchEventItems[endpointId] = { Endpoint:{}, Events: {} }; const endpoint = event.Endpoints[endpointId]; endpoint.RecommendationItems.forEach((itemId) => { const eventKey = `${endpointId}_rec_${itemId}`; batchEventItems[endpointId].Events[eventKey] = { EventType: '_custom.recommender', Timestamp: new Date().toISOString(), Attributes: { personalize_user_id: endpointId || endpoint.User.UserId, // TODO Determine UserId or EndpointId campaign_id: event.CampaignId, item_id: itemId } }; }); }); console.log(JSON.stringify(batchEventItems, 1)); return pinpoint.putEvents({ ApplicationId: event.ApplicationId, EventsRequest: { BatchItem: batchEventItems } }).promise(); };

    The following code is an example Python implementation.

    # Example Python Lambda Function import json import boto3 import datetime pinpoint = boto3.client('pinpoint') def lambda_handler(event, context): emit_recommender_events(event) # ... normal processing ... return event['Endpoints'] def emit_recommender_events(event): batchEventItems = {} for endpointId in event['Endpoints'] : batchEventItems[endpointId] = { 'Endpoint': {}, 'Events': {} } endpoint = event['Endpoints'][endpointId] for itemId in endpoint['RecommendationItems'] : eventKey = endpointId + '_rec_' + itemId batchEventItems[endpointId]['Events'][eventKey] = { 'EventType': '_custom.recommender', 'Timestamp': datetime.datetime.now().isoformat(), 'Attributes': { 'personalize_user_id': endpointId || endpoint['User']['UserId'], # TODO Determine UserId or EndpointId, 'campaign_id': event['CampaignId'], 'item_id': itemId } } pinpoint.put_events( ApplicationId=event['ApplicationId'], EventsRequest={ 'BatchItem': batchEventItems } )
  5. Scroll to the Execution role section and, under Existing role, choose the View the <function-name> role on the IAM console text link. The IAM console opens in a new browser tab.

  6. On the Summary page, choose Attach policies.

  7. On the Add permissions page, choose Create policy. The Create policy page opens in a new browser tab.

  8. In the Visual editor tab, Service option, choose the Choose a service text link and in the search field enter Pinpoint.

  9. Select Pinpoint from the search results.

  10. For Actions, choose the drop-down arrow next to Write to expand the list of options and select PutEvents.

  11. Choose the drop-down arrow next to Resources to access the list of options and then select All resources.

  12. Choose Review policy.

  13. In the Name field, enter a <policy-name> and, optionally, enter a Description.

    Note

    For the <policy-name>, create a name for this role that is meaningful to you, for example, EmitCustomEventsPolicy.

  14. Choose Create policy.

  15. Return to the Add permissions page, and choose the refresh button to ensure that your newly created policy is added to the list, and search for that policy.

  16. Select your policy and choose Attach policy.

You return to the IAM policy page and receive a confirmation that the policy was attached. Optionally, you can enter the JSON code to add permission to the role that runs the Lambda function. For instructions, see Appendix A

Step 3. Verify Updated Amazon Personalize Campaign

The AWS Step Functions state machine queries the newly imported interaction dataset, retrains Amazon Personalize to create a new version of the personalized solution, and updates the Amazon Personalize campaign. You can verify that Amazon Personalize was updated with a new version of the personalized solution by taking the following steps.

Note

The state machine is configured to run every morning at 2:00 AM GMT. The state machine does not make changes or updates to Amazon Personalize until new interaction data is received. When new data is available, the state machine queries and retrains Amazon Personalize as needed.

  1. Navigate to the Amazon Personalize console.

  2. Select your dataset group in the Dataset groups list.

  3. In the navigation pane, choose Datasets then under Dataset type, locate User-item interaction and choose the corresponding name under Dataset name.

  4. Identify the latest import job and verify that the Import job status is Active.

  5. In the navigation pane, choose Solutions and recipes and select the solution that is in use by the campaign.

  6. Identify the latest Solution version and verify that the Solution version status is Active. Note the Solution version ID. You will match the ID in the following steps when you navigate to the Campaigns page.

  7. In the navigation pane, choose Campaigns, select your campaign and then select the Details tab.

  8. Verify that the campaign's Solution version ID matches the ID listed in the Solution screen.

As you execute Amazon Pinpoint campaigns based on the Amazon Personalize Recommender model integration, this solution will track user interactions and provide new data to Amazon Personalize daily. This allows your recommendations to improve over time as Amazon Personalize learns with the goal of driving higher end user engagements.

This solution can be extended to consume and score custom events from other applications based on the item being recommended. For example, track the recommended item from Amazon Personalize to purchase events to use for retraining Amazon Personalize. You can create custom events and submit them to Amazon Pinpoint as detailed in Step 2.