Tutorial: Using the AWS Glue Connector for Elasticsearch
Elasticsearch is a popular open-source search and analytics engine for use cases such as log
analytics, real-time application monitoring, and clickstream analysis. You can use OpenSearch as a
data store for your extract, transform, and load (ETL) jobs by configuring the
AWS Glue Connector for Elasticsearch in AWS Glue Studio. This connector
is available for free from
AWS Marketplace
Note
The AWS Marketplace
Elasticsearch Spark Connector
In this tutorial, we will show how to connect to your Amazon OpenSearch Service nodes with a minimal number of steps.
Topics
- Prerequisites
- Step 1: (Optional) Create an AWS secret for your OpenSearch cluster information
- Step 2: Subscribe to the connector
- Step 3: Activate the connector in AWS Glue Studio and create a connection
- Step 4: Configure an IAM role for your ETL job
- Step 5: Create a job that uses the OpenSearch connection
- Step 6: Run the job
Prerequisites
To use this tutorial, you must have the following:
-
Access to AWS Glue Studio
-
Access to an OpenSearch cluster in the AWS Cloud
-
(Optional) Access to AWS Secrets Manager.