Natural language query generation with OpenSearch - Amazon OpenSearch Service
PrerequisitesGetting startedConfigure permissionsConfiguration automation

Natural language query generation with OpenSearch

The natural language query generation feature in Amazon OpenSearch Service allows you to query your operational and security log data through natural language. OpenSearch is an ideal option to explore log data because it is a highly scalable and performant log analytics and search engine, and now you can use natural language to explore these logs. This feature allows you to identify issues without relying on OpenSearch Piped Processing Language (PPL) or having to look up data definitions when you build your queries. You can use the natural language query generation feature on OpenSearch Service domains with version 2.13 and later. You must have fine-grained access control enabled.

This feature was built with the OpenSearch Assistant Toolkit. If you want to create similar features that connect to your large language models, you can use the toolkit to configure your own agents and tools.

Prerequisites

Before you can use the natural language query generation feature, your domain must have the following:

Getting started

To start using the natural language query generation feature, make sure you have the feature enabled on your OpenSearch Service domain. This feature is enabled by default on all domains created with version 2.13 and later that have fine-grained access control enabled.

If you upgraded to OpenSearch version 2.13 before July 2, 2024, you must update your service software to (R20240520-P4) or later before you can enable natural language query generation. After you do this, you can enable the feature by selecting the Enable natural language query generation box checkbox under the Artificial Intelligence (AI) and Machine Learning (ML) section.

After you have your domain set up, navigate to the Log Explorer page in OpenSearch Dashboards. Choose Event Explorer and ask a question with the query assistant.

Configure permissions

If you enable natural language query generation on a preexisting OpenSearch Service domain, the query_assistant_access role might not be defined on the domain. Non-admin users must be mapped to this role in order to manage warm indexes on domains using fine-grained access control. To manually create the query_assistant_access role, perform the following steps:

  1. In OpenSearch Dashboards, go to Security and choose Roles.

  2. Choose Create role and configure the following cluster permissions:

    • cluster:admin/opensearch/ml/config/get

    • cluster:admin/opensearch/ml/execute

    • cluster:admin/opensearch/ml/predict

    • cluster:admin/opensearch/ppl

  3. Name the role query_assistant_access.

  4. Choose Create role. The query_assistant_access role is now available.

    Note

    You must also have the indices:admin/mappings/get and read index permissions for the indices that you want to use natural language questions with.

Configuration automation

Flow Framework is an OpenSearch plugin that provides a way to automate OpenSearch configurations for use cases such as query generation and conversational chat. Because the plugin tracks the resources that enable the natural language query generation feature, the flow framework index stores a template for each domain that uses query assist.

Flow Framework allows you to either select from a set of predefined templates, or create your own automations for machine learning connectors, tools, agents, and other components that prepare OpenSearch as a backend for generative models.