Getting batch recommendations
To get recommendations for large datasets that do not require real-time updates, you can use an asynchronous batch workflow with historical data only. For example, you might get product recommendations for all users on an email list, or get item-to-item similarities (SIMS) across an inventory.
To get batch recommendations with Amazon Personalize, you use a batch inference job. A batch inference job is a tool that imports your batch input data from an Amazon S3 bucket, uses your solution version to generate recommendations, and exports the recommendations to an Amazon S3 bucket. We recommend that you use a different location for your output data (either a folder or a different Amazon S3 bucket). To use data you record in real time with the PutEvents API operation, you must retrain your solution version before creating the batch inference job.
The batch workflow is as follows:
-
Prepare and upload a list of users or items (or both) in JSON format to an Amazon S3 bucket. The format of your input data depends on the recipe you use. See Preparing and importing batch input data.
-
Create a separate location for your output data, either a folder or a different Amazon S3 bucket.
-
Create a batch inference job. See Creating a batch inference job (console), Creating a batch inference job (AWS CLI), or Creating a batch inference job (AWS SDKs).
-
When the batch inference job is complete, retrieve the recommendations from your output location in Amazon S3.
Topics
Batch workflow permissions requirements
For batch workflows, your Amazon Personalize IAM service role needs permission to access and add files to your Amazon S3 buckets. For information on granting permissions, see Service-linked role policy for batch workflows. For more information on bucket permissions, see User policy examples in the Amazon Simple Storage Service (S3) Developer Guide.
Amazon S3 buckets and objects must be either encryption free or, if you are using AWS Key Management Service (AWS KMS) for encryption, you must give your IAM user and Amazon Personalize service-linked role permission to use your key. You must also add Amazon Personalize as a Principle in your AWS KMS key policy. For more information, see Using key policies in AWS KMS in the AWS Key Management Service Developer Guide.
Batch workflow scoring
Item scores calculated by batch inference jobs are calculated just as described in Getting real-time recommendations. You can view scores in the batch inference job's output JSON file. Scores are only returned by models trained with the HRNN and Personalize-Ranking recipes.
Preparing and importing batch input data
A batch inference job uses a solution version to make recommendations based on the users or items (or both) that you provide in an input JSON file. Before you can get batch recommendations, you must prepare and upload your JSON file to an Amazon S3 bucket. We recommend that you create an output folder in your Amazon S3 bucket or use a separate output Amazon S3 bucket. This allows you to run multiple batch inference jobs using the same input data location.
To prepare and import data
-
Format your batch input data depending on the recipe your solution uses. Your input data is a JSON file with either a list of userIds (USER_PERSONALIZATION recipes), a list of itemIds (RELATED_ITEMS recipes), or both (PERSONALIZED_RANKING recipes). Separate each user, item, or user and list of items with a new line. For examples, see Input and output JSON examples.
-
Upload your input JSON to an input folder in your Amazon S3 bucket. For more information, see Uploading files and folders by using drag and drop in the Amazon Simple Storage Service Console User Guide.
-
Create a separate location for your output data, either a folder or a different Amazon S3 bucket. Creating a separate location for the output JSON allows you to run multiple batch inference jobs using the same input data location.
Input and output JSON examples
How you format your input data depends on the recipe you use. The following are correctly formatted JSON input and output examples for each recipe type.
User-Personalization and legacy HRNN recipes
Popularity-Count
Personalize-Ranking
SIMS
Creating a batch inference job (console)
After you have completed Preparing and importing batch input data, you are ready to create a batch inference job. This procedure assumes that you have already created a solution and a solution version (trained model). To get batch recommendations, create a batch inference job and specify the job name,
To get batch recommendations (console)
-
Open the Amazon Personalize console at https://console.aws.amazon.com/personalize/home
and sign in to your account. -
Choose Batch inference jobs in the navigation pane, then choose Create batch inference job.
-
In Batch inference job details, in Batch inference job name, specify a name for your batch inference job.
-
For IAM service role, choose the IAM service role you created for Amazon Personalize during set up. This role must have read and write access to your input and output Amazon S3 buckets respectively.
-
For Solution, choose the solution and then choose the Solution version ID that you want to use to generate the recommendations.
-
For Input data configuration, specify the Amazon S3 path to your input file. Your input data must be in the correct format for the recipe your solution uses. For input data examples see Input and output JSON examples.
-
For Output data configuration, specify the path to your output location. We recommend using a different location for your output data (either a folder or a different Amazon S3 bucket).
-
For Filter configuration optionally choose a filter to apply a filter to the recommendations added to the output JSON file. For more information see Filtering batch recommendations.
-
Choose Create batch inference job. Batch inference job creation starts and the Batch inference jobs page appears with the Batch inference job detail section displayed.
-
When the batch inference job's status changes to Active, you can retrieve the job's output from the designated output Amazon S3 bucket. The output file's name will be of the format
input-name.out
Creating a batch inference job (AWS CLI)
After you have completed Preparing and importing batch input data, you are ready to
create a batch inference job using the following create-batch-inference-job code.
Specify a job name, replace Solution version ARN with the Amazon Resource Name (ARN) of your solution version,
and replace the IAM service role ARN with the ARN of the IAM service role you created for Amazon Personalize during set
up. This role must have read and write access to your input and output Amazon S3 buckets
respectively.
Replace S3 input path and S3 output path with the Amazon S3 path to your input file and output locations. We recommend
using a different location for your output data (either a folder or a different Amazon
S3 bucket). You can apply
a filter to the recommendations added to the output JSON file. For more information
see Filtering batch recommendations.
The example includes optional User-Personalization recipe specific
itemExplorationConfig hyperparameters: explorationWeight and
explorationItemAgeCutOff. Optionally include explorationWeight and
explorationItemAgeCutOff values to configure exploration.
For more information, see
User-Personalization recipe.
aws personalize create-batch-inference-job --job-nameBatch job name\ --solution-version-arnSolution version ARN\ --job-input s3DataSource={path=s3://S3 input path} \ --job-output s3DataDestination={path=s3://S3 output path} \ --role-arnIAM service role ARN\ --batch-inference-job-config "{\"itemExplorationConfig\":{\"explorationWeight\":\"0.3\",\"explorationItemAgeCutOff\":\"30\"}}"{ "batchInferenceJobArn": "arn:aws:personalize:us-west-2:012345678901:batch-inference-job/batchTest" }
Creating a batch inference job (AWS SDKs)
After you have completed Preparing and importing batch input data, you are ready to create a batch inference job with the CreateBatchInferenceJob operation. The following code shows how to create a batch inference job using the AWS SDK for Python (Boto3) or AWS SDK for Java 2.x.
Processing the batch job might take a while to complete. You can check a job's status
by
calling DescribeBatchInferenceJob and passing a
batchRecommendationsJobArn as the input parameter. You can also list all
Amazon Personalize batch inference jobs in your AWS environment by calling ListBatchInferenceJobs.
