Creating and connecting to a Timestream for InfluxDB instance - Amazon Timestream
PrerequisitesStep 1: Create an Amazon EC2 instanceStep 2: Create an InfluxDB DB instanceStep 3: Send Telegraf data to your InfluxDB instanceStep 4: Delete the Amazon EC2 instance and the InfluxDB DB instance

Creating and connecting to a Timestream for InfluxDB instance

This tutorial creates an Amazon EC2 instance and an Amazon Timestream for InfluxDB DB instance. The tutorial shows you how to write data to the DB instance from the EC2 instance using the Telegraf client. As a best practice, this tutorial creates a private DB instance in a virtual private cloud (VPC). In most cases, other resources in the same VPC, such as EC2 instances, can access the DB instance, but resources outside of the VPC can't access it.

After you complete the tutorial, there is a public and private subnet in each Availability Zone in your VPC. In one Availability Zone, the EC2 instance is in the public subnet, and the DB instance is in the private subnet.

Note

There's no charge for creating an AWS account. However, by completing this tutorial, you might incur costs for the AWS resources you use. You can delete these resources after you complete the tutorial if they are no longer needed.

The following diagram shows the configuration when accessibility is public.

Warning

We don't recommend using 0.0.0.0/0 for HTTP access, since you make it possible for all IP addresses to access your public InfluxDB instance via HTTP. This approach is not even acceptable for a short time in a test environment. Authorize only a specific IP address or range of addresses to access your InfluxDB instances using HTTP being for WebUI or API access.

This tutorial creates a DB instance running InfluxDB with the AWS Management Console. We will focus only on the DB instance size and DB instance identifier. We will use the default settings for the other configuration options. The DB instance created by this example will be private.

Other settings that you could configure include availability, security, and logging. To create a public DB instance, you must choose to make your instance “Publicly accessible” on the Connectivity configuration section. For information about creating DB instances, see Creating a DB instance..

If your instance is not publicly accessible, do the following:

  • Create a host on the VPC of the instance through which you can tunnel traffic.

  • Setup ssh tunneling to the instance. For more information, see Amazon EC2 instance port forwarding with AWS Systems Manager

  • In order for the certificate to work, add the following line to the /etc/hosts file of your client machine: 127.0.0.1. This is the DNS address of your instance.

  • Connect to your instance using the fully qualified domain name, for example, https://<DNS>:8086.

    Note

    Localhost is unable to validate the certificate because localhost is not part of the certificate SAN.

The following diagram shows the configuration when accessibility is private:

Prerequisites

Before you begin, complete the steps in the following sections:

  • Sign up for an AWS account.

  • Create an administrative user.

Step 1: Create an Amazon EC2 instance

Create an Amazon EC2 instance that you will use to connect to your database.

  1. Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. In the upper-right corner of the AWS Management Console, choose the AWS Region in which you want to create the EC2 instance.

  3. Choose EC2 Dashboard, and then choose Launch instance.

  4. When the Launch an instance page opens, choose the following settings on the Launch an instance page.

    1. Under Name and tags, for Name, enter ec2-database-connect.

    2. Under Application and OS Images (Amazon Machine Image), choose Amazon Linux, and then choose the Amazon Linux 2023 AMI. Keep the default selections for the other choices.

    3. Under Instance type, choose t2.micro.

    4. Under Key pair (login), choose a Key pair name to use an existing key pair. To create a new key pair for the Amazon EC2 instance, choose Create new key pair and then use the Create key pair window to create it. For more information about creating a new key pair, see Create a key pair in the Amazon EC2 User Guide for Linux Instances.

    5. For Allow SSH traffic in Network settings, choose the source of SSH connections to the EC2 instance. You can choose My IP if the displayed IP address is correct for SSH connections. Otherwise, you can determine the IP address to use to connect to EC2 instances in your VPC using Secure Shell (SSH). To determine your public IP address, in a different browser window or tab, you can use the service at https://checkip.amazonaws.com. An example of an IP address is 192.0.2.1/32. In many cases, you might connect through an internet service provider (ISP) or from behind your firewall without a static IP address. If so, make sure to determine the range of IP addresses used by client computers.

      Warning

      We don’t recommend using 0.0.0.0/0 for SSH access, since you make it possible for all IP addresses to access your public EC2 instances using SSH. This approach is not even acceptable for a short time in a test environment, authorize only a specific IP address or range of addresses to access your EC2 instances using SSH.

Step 2: Create an InfluxDB DB instance

The basic building block of Amazon Timestream for InfluxDB is the DB instance. This environment is where you run your InfluxDB databases.

In this example, you will create a DB instance running the InfluxDB database engine with a db.influx.large DB instance class.

  1. Sign in to the AWS Management Console and open the Amazon Timestream for InfluxDB console at https://console.aws.amazon.com/timestream/.

  2. In the upper-right corner of the Amazon Timestream for InfluxDB console, choose the AWS Region in which you want to create the DB instance.

  3. In the navigation pane, choose InfluxDB Databases.

  4. Choose Create Influx database.

  5. For DB Instance Identifier, enter KronosTest-1.

  6. Provide the InfluxDB basic configuration parameters: User Name, Organization, Bucket Name and Password.

    Important

    You won't be able to view the user password again. You won't be able to access your instance and obtain an operator token without your password. If you don't record it, you might have to change it. See Creating a new Operator Token for your InfluxDB instance.

    If you need to change the user password after the DB instance is available, you can modify the DB instance to do so. For more information about modifying a DB instance, see Updating DB instances.

  7. For DB Instance Class, select db.influx.large.

  8. For DB Storage Class, select influx IOPS Included 3K.

  9. Configure your logs. For more information, see Setup to view InfluxDB Logs on Timestream Influxdb Instances.

  10. In the Connectivity configuration section, make sure your InfluxDB instance is in the same subnet as your newly created EC2 instance.

  11. Choose Create Influx database.

  12. In the Databases list, chose the name of your new InfluxDB instance to show its details. The DB Instance has a status of Creating until it is ready to use.

You can connect to the DB instance when the status changes to Available. Depending on the DB instance class and the amount of storage, it can take up to 20 minutes before the new instance is available.

Important

At this time, you can't modify compute (Instance types) and Storage (Storage Types) configuration of existing instances.

Step 3: Send Telegraf data to your InfluxDB instance

You can now start sending telemetry data to your InfluxDB DB instance using the Telegraf agent. In this example, you'll install and configure a Telegraf agent to send performance metrics to you InfluxDB DB instance.

  1. Find the endpoint (DNS name) and port number for your DB instance.

    1. Sign in to the AWS Management Console and open the Amazon Timestream console at https://console.aws.amazon.com/timestream/.

    2. In the upper-right corner of the Amazon Timestream console, choose the AWS Region for the DB instance.

    3. In the navigation pane, choose InfluxDB Databases.

    4. Choose the InfluxDB DB instance name to display its details.

    5. On the Summary section, copy the endpoint. Also note the port number. You need both the endpoint and the port number to connect to the DB instance (the default port number for InfluxDB is 8086).

  2. Next, select InfluxDB UI.

  3. This will open a new browser window where you should see a login prompt. Enter the credentials you used earlier to create your InfluxDB Db instance.

  4. In the navigation pane, click on the Arrow and select API Tokens.

  5. For this test, generate an All Access Token.

    Note

    For production scenarios, we recommend creating tokens with specific access to the required buckets that are built for specific Telegraf needs.

  6. Your Token will appear on the screen.

    Important

    Make sure to copy and save the Token since you won’t be able to display it again.

  7. Connect to the EC2 instance that you created earlier by following the steps in Connect to your Linux instance in the in the Amazon EC2 User Guide for Linux Instances.

    We recommend that you connect to your EC2 instance using SSH. If the SSH client utility is installed on Windows, Linux, or Mac, you can connect to the instance using the following command format:

    ssh -i location_of_pem_file ec2-user@ec2-instance-public-dns-name

    For example, assume that ec2-database-connect-key-pair.pem is stored in /dir1 on Linux, and the public IPv4 DNS for your EC2 instance is ec2-12-345-678-90.compute-1.amazonaws.com. Your SSH command would look as follows:

    ssh -i /dir1/ec2-database-connect-key-pair.pem ec2-user@ec2-12-345-678-90.compute-1.amazonaws.com

  8. Get the latest version of telegraf installed on your instance. To do this, use the following command:

    cat <<EOF | sudo tee /etc/yum.repos.d/influxdata.repo
[influxdata]
name = InfluxData Repository - Stable
baseurl = https://repos.influxdata.com/stable/\$basearch/main
enabled = 1
gpgcheck = 1
gpgkey = https://repos.influxdata.com/influxdata-archive_compat.key
EOF

sudo yum install telegraf

  9. Configure your Telegraf instance.

    Note

    If telegraf.conf does not exist or contain a timestream section, you can generate one with:

    telegraf —section-filter agent:inputs:outputs —input-filter cpu:mem —output-filter timestream config > telegraf.conf

    1. Edit the configuration file usually located at /etc/telegraf.

      sudo nano /etc/telegraf/telegraf.conf

    2. Configure basic inputs for CPU, MEM and DISK.

      [[inputs.cpu]]
  percpu = true
  totalcpu = true
  collect_cpu_time = false
  report_active = false

[[inputs.mem]]

[[inputs.disk]]
  ignore_fs = ["tmpfs", "devtmpfs", "devfs"]

    3. Configure the Output plug in to send data to your InfluxDB DB instance and save your changes.

      [[outputs.influxdb_v2]]
   urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
   token = "<your_telegraf_token"
   organization = "your_org"
   bucket = "your_bucket"
   timeout = "5s"

    4. Configure the Timestream target.

      # Configuration for sending metrics to Amazon Timestream.
[[outputs.timestream]]

  ## Amazon Region and credentials
  region = "us-east-1"
  access_key = "<AWS key here>"
  secret_key = "<AWS secret key here>"
  database_name = "<timestream database name>" # needs to exist

  ## Specifies if the plugin should describe t start.
  describe_database_on_start = false
  mapping_mode = "multi-table" # allows multible tables for each input metrics

  create_table_if_not_exists = true
  create_table_magnetic_store_retention_period_in_days = 365
  create_table_memory_store_retention_period_in_hours = 24

  use_multi_measure_records = true # Important to use multi-measure records
  measure_name_for_multi_measure_records = "telegraf_measure"
  max_write_go_routines = 25

  10. Enable and start the Telegraf service.

    $ sudo systemctl enable telegraf
$ sudo systemctl start telegraf

Step 4: Delete the Amazon EC2 instance and the InfluxDB DB instance

After you explore the Telegraf-generated data using your your InfluxDB DB instance with the InfluxUI, delete both your EC2 and your InfluxDB DB instances so you are no longer charged for them.

To delete the EC2 instance:

  1. Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. In the navigation pane, choose Instances.

  3. Select the EC2 instance, choose Instance state, and Terminate instance.

  4. Choose Terminate when prompted for confirmation.

For more information about deleting an EC2 instance, see Terminate your instance in the Amazon EC2 User Guide.

To delete the DB instance with no final DB snapshot:

  1. Sign in to the AWS Management Console and open the Amazon Timestream for InfluxDB console at https://console.aws.amazon.com/timestream/.

  2. In the navigation pane, choose InfluxDB Databases.

  3. Choose the DB instance you want to delete.

  4. For Actions, choose Delete.

  5. Complete the acknowledgement and choose Delete.

(Optional) Connect to your DB instance using Amazon Managed Grafana

You can use Amazon Managed Grafana to create dashboards and monitor the performance of your EC2 instances using Amazon Timestream for InfluxDB. Amazon Managed Grafana is a fully managed service for Grafana, a popular open-source analytics platform that enables you to query, visualize, and alert on your metrics, logs, and traces.