Tutorial: Using the AWS Glue Connector for Elasticsearch

Elasticsearch is a popular open-source search and analytics engine for use cases such as log analytics, real-time application monitoring, and clickstream analysis. You can use OpenSearch as a data store for your extract, transform, and load (ETL) jobs by configuring the AWS Glue Connector for Elasticsearch in AWS Glue Studio. This connector is available for free from AWS Marketplace.

Note

The AWS Marketplace Elasticsearch Spark Connector has been deprecated. Please use the AWS Glue Connector for Elasticsearch instead.

In this tutorial, we will show how to connect to your Amazon OpenSearch Service nodes with a minimal number of steps.

Topics

• Prerequisites
• Step 1: (Optional) Create an AWS secret for your OpenSearch cluster information
• Step 2: Subscribe to the connector
• Step 3: Activate the connector in AWS Glue Studio and create a connection
• Step 4: Configure an IAM role for your ETL job
• Step 5: Create a job that uses the OpenSearch connection
• Step 6: Run the job

Prerequisites

To use this tutorial, you must have the following:

• Access to AWS Glue Studio

• Access to an OpenSearch cluster in the AWS Cloud

• (Optional) Access to AWS Secrets Manager.