Personalizing search results from OpenSearch

You can use Amazon Personalize to personalize results from open source OpenSearch or Amazon OpenSearch Service for your users.

OpenSearch is a self-managed, open source search service based on the Apache 2.0 License. Amazon OpenSearch Service is a managed service that helps you deploy, operate, and scale OpenSearch resources in the AWS Cloud. When you use Amazon OpenSearch Service, OpenSearch retrieves and ranks results.

When ranking query results, OpenSearch uses a probabilistic ranking framework called BM-25 to calculate relevance scores. If a distinctive keyword appears more frequently in a document, BM-25 assigns a higher relevance score to that document. OpenSearch ranking doesn’t take into account user behavior like click-through data.

When you use Amazon Personalize with OpenSearch, Amazon Personalize re-ranks OpenSearch results based on a user's past behavior, any metadata about the items, and any metadata about the user. OpenSearch then incorporates the re-ranking before returning the search response to your application. You control how much weight OpenSearch gives the ranking from Amazon Personalize when applying it to OpenSearch results.

With this re-ranking, results can be more engaging and relevant to a user's interests. This can lead to an increase in the click-through rate and conversion rate for your application. For a use case example that describes how personalized search can improve results for an ecommerce application, see Use case example.

Before you start personalizing OpenSearch results, review the requirements listed in Guidelines and requirements.

Topics

• Use case example
• Personalized search workflow
• How the Amazon Personalize Search Ranking plugin works
• Additional information
• Guidelines and requirements
• Setting up OpenSearch and installing the plugin
• Configuring the plugin
• Applying the plugin to OpenSearch queries
• Comparing OpenSearch results with results from the plugin
• Monitoring the plugin

Use case example

When you use Amazon Personalize to re-rank OpenSearch results, the search results can be more relevant for your users. For example, you might have an ecommerce application that sells cars. If your user enters a query for Toyota cars and you don't personalize results, OpenSearch would return a list of cars made by Toyota based on keywords in your data. This list would be ranked in the same order for all users.

But if you use Amazon Personalize to personalize results, OpenSearch re-ranks these cars in order of relevance for the specific user based on their behavior—for example, their clicks. The car that the user is most likely to click is ranked first.

When you personalize OpenSearch results, you control how much weight (emphasis) OpenSearch gives the ranking from Amazon Personalize. Continuing with this example, if a user searches for a specific type of car from a specific year (such as a 2008 Toyota Prius), you might want to put more emphasis on the original ranking from OpenSearch.

However, for more generic queries that result in a wide range of results (such as a search for all Toyota vehicles), you might put a high emphasis on personalization. This way, the cars at the top of the list are more relevant to the particular user.

Personalized search workflow

To personalize OpenSearch results, you do the following:

1. Set up Amazon Personalize – If you haven't already, complete the steps in Setting up Amazon Personalize to set up your credentials and set up permissions for Amazon Personalize. You don't need to set up the AWS SDKs to personalize OpenSearch results.

2. Complete the Amazon Personalize workflow – Complete the Amazon Personalize workflow to import data, create a solution with the Personalized-Ranking recipe, train a custom solution version, and deploy it in a campaign. You can only use the Personalized-Ranking recipe. You must create an Item interactions dataset. A Users dataset and an Items dataset are optional. For more information, see Amazon Personalize workflow.

3. Set up OpenSearch and install the Amazon Personalize Search Ranking plugin – If you haven't already, set up your OpenSearch Service domain or open source OpenSearch cluster. Then install the Amazon Personalize Search Ranking plugin. This plugin handles communication with Amazon Personalize and re-ranking results. For more information, see Setting up OpenSearch and installing the plugin.

4. Configure the Amazon Personalize Search Ranking plugin – To configure the plugin, you create search pipelines. Search pipelines are sets of request and response processors. When you create a pipeline for the plugin, you specify your Amazon Personalize resources in a personalized_search_ranking response processor. You also configure how much weight the plugin gives the results from Amazon Personalize when it re-ranks results. For more information, see Configuring the plugin.

5. Apply the Amazon Personalize Search Ranking plugin to OpenSearch queries – You can apply the Amazon Personalize Search Ranking plugin to all queries and responses for an OpenSearch index. You can also apply the plugin to individual OpenSearch queries. For more information, see Applying the plugin to OpenSearch queries.

6. Compare results – The Amazon Personalize Search Ranking plugin re-ranks the search results in the OpenSearch query response. It considers both the ranking from Amazon Personalize and the ranking from OpenSearch. To understand how results are re-ranked, you can compare results from queries that use personalization and those that don't. For more information, see Comparing OpenSearch results with results from the plugin.

7. Monitor the Amazon Personalize Search Ranking plugin – As you apply the Amazon Personalize Search Ranking plugin to search queries, you can monitor the plugin by getting metrics for your search pipelines. For more information, see Monitoring the plugin.

How the Amazon Personalize Search Ranking plugin works

The following diagram shows how the Amazon Personalize Search Ranking plugin works.

1. You submit your customer's query to your OpenSearch Service domain or your open source OpenSearch cluster.

2. OpenSearch sends the query response (list of items that are relevant to the query) and the user's ID to the Amazon Personalize Search Ranking plugin.

3. The plugin sends the items and user in the response to your Amazon Personalize campaign for ranking. It uses the recipe and campaign Amazon Resource Name (ARN) values in your search pipeline to get a personalized ranking for the user. It uses the GetPersonalizedRanking API operation for recommendations. In the request, it passes the userId of the user making the query and the items returned from the OpenSearch query in the inputList.

4. Amazon Personalize returns the re-ranked results to the plugin.

5. The plugin rearranges and returns the search results to your OpenSearch Service domain or open source OpenSearch cluster. It re-ranks the results based on the response from your Amazon Personalize campaign and the emphasis on personalization that you specify during setup.

6. Your open source OpenSearch cluster or OpenSearch Service domain returns the final results to your application.

Additional information

The following resources provide additional information about using OpenSearch.

• For information about getting started with open source OpenSearch, see Quickstart.

• For information about getting started with OpenSearch Service, see Getting started with Amazon OpenSearch Service in the Amazon OpenSearch Service Developer Guide.

• For information about the Personalized-Ranking recipe in Amazon Personalize, see Personalized-Ranking recipe.