Configuring your workload instances

The dashboard deployed by the solution assumes that your instances are configured correctly and send the appropriate metrics and logs to Amazon CloudWatch. If your EC2 instances do not send metrics and logs in the appropriate format, they will not appear on the dashboard.

The demo instances created by the workload demo templates are pre-configured with these configurations and can be used for reference purpose. Refer to AWS CloudFormation templates for details on demo templates.

Note: You can use AWS Systems Manager run command to configure your instances (highlighted below in Configuring EC2 instance sections). For more information, refer to AWS Systems Manager Run Command in the AWS Systems Manager User Guide.

Configuring your Apache EC2 instance

For more information about metrics and logs, refer to Apache key performance indicators.

Amazon CloudWatch agent configuration for Apache

Refer to the sample configuration files apache.json and linux_cw_infra.json to ensure that your EC2 instance CloudWatch agent is sending metrics and logs in the appropriate format.

The following example code is from the apache.json file. This code identifies the appropriate process-level metrics being sent to Amazon CloudWatch.

"metrics_collected": {
      "procstat": [
        {
          "exe": "httpd",
          "measurement": [
            "cpu_usage",
            "memory_rss",
            "memory_vms",
            "read_bytes",
            "write_bytes",
            "read_count",
            "write_count"
          ]
        }
      ]    

Similarly, the following example code is also from the apache.json and it identifies the web server access log file path and corresponding CloudWatch log group name. The CloudWatch log group name should be consistent with Access Log Group parameter value in the workload template.

"logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/var/log/www/error/*",
            "log_group_name":"/cw-monitoring-framework/apache/error",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/var/log/www/access/*",
            "log_group_name": "/cw-monitoring-framework/apache/access",
            "log_stream_name": "{instance_id}"
          }
        ]
      }     

httpd.conf for Apache web server

The following example code shows the appropriate log format for the Apache access log in the httpd.conf file to support CloudWatch dashboard queries. You can modify this file to customize log data that you want to display on your Apache dashboard. Additionally, the data points and schema in this file must be consistent with ApacheLogWidgets in the widget export file.

httpd.conf

Configuring your NGINX EC2 instance

For more information about metrics and logs, refer to NGINX key performance indicators.

Amazon CloudWatch agent configuration for NGINX

Refer to the sample configuration files, nginx.json, and linux_cw_infra.json to ensure that your EC2 instance CloudWatch agent is sending metrics and logs in the appropriate format.

The following example code is from the nginx.json file. This code identifies the appropriate process-level metrics being sent to Amazon CloudWatch.

“metrics_collected”: {
      “procstat”: [
        {
          “exe”: “nginx”,
          “measurement”: [
            “cpu_usage”,
            “memory_rss”,
            “memory_vms”,
            “read_bytes”,
            “write_bytes”,
            “read_count”,
            “write_count”
          ]
        }
      ]   

Similarly, the following example code is also from the nginx.json. It identifies the web server access log file path and corresponding CloudWatch log group name. The CloudWatch log group name should be consistent with Access Log Group parameter value in the workload template.

“logs”: {
    “logs_collected”: {
      “files”: {
        “collect_list”: [
          {
            “file_path”: “/var/log/nginx/access.log”,
            “log_group_name”: “/cw-monitoring-framework/nginx/access”,
            “log_stream_name”: “{instance_id}”
          },
          {
            “file_path”: “/var/log/nginx/error.log”,
            “log_group_name”: “/cw-monitoring-framework/nginx/error”,
            “log_stream_name”: “{instance_id}”
          }
        ]
      }
    }
  }   

nginx.conf for NGINX web server

The following example code shows the appropriate log format for the NGNIX access log in the nginx.conf file, to support CloudWatch dashboard queries. You can modify this file to customize log data that you want to display on your NGINX dashboard. Additionally, the data points and schema in this file must be consistent with NginxLogWidgets in the widget export file.

nginx.conf

Configuring your Puma EC2 instance

For more information about metrics and logs, refer to Puma key performance indicators.

Amazon CloudWatch agent configuration for Puma

Refer to the sample configuration files puma.json and linux_cw_infra.json to ensure that your EC2 instance CloudWatch agent is sending metrics and logs in the appropriate format.

The following example code is from the puma.json file. This code identifies the appropriate statsd and process-level metrics being sent to Amazon CloudWatch.

"metrics_collected": {
      "statsd": {
        "service_address": ":8125",
        "metrics_collection_interval": 10,
        "metrics_aggregation_interval": 10
      },
      "procstat": [
        {
          "exe": "ruby",
          "measurement": [
            "cpu_usage",
            "memory_rss",
            "memory_vms",
            "read_bytes",
            "write_bytes",
            "read_count",
            "write_count",
            "pid_count",
            "realtime_priority",
            "nice_priority"
          ]
        }
      ]
    }

Similarly, the following example code is also from the puma.json. It identifies the web server access log file path and corresponding CloudWatch log group name. The CloudWatch log group name should be consistent with Access Log Group parameter value in the workload template.

"logs": {
    "logs_collected": {
      "files": {
        "collect_list": [
          {
            "file_path": "/root/sample-app/log/puma.stderror.log",
            "log_group_name": "/cw-monitoring-framework/puma/stderror",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/root/sample-app/log/puma.stdout.log",
            "log_group_name": "/cw-monitoring-framework/puma/out",
            "log_stream_name": "{instance_id}"
          },
          {
            "file_path": "/root/sample-app/log/production.log",
            "log_group_name": "/cw-monitoring-framework/puma/access",
            "log_stream_name": "{instance_id}"
          }
        ]
      }
    }
  }    

Configurations for Puma web server

Following configuration files are used to bootstrap the demo Puma instance as well.

• puma.rb: The following code example shows how to configure stdout and stderr logs on the Puma web server. The log file paths should be consistent with the file paths in CloudWatch agent configurations. Additionally, statsd plugin needs to be turned on as well.

#define location of stdout and stderr logs 
stdout_redirect "/root/sample-app/log/puma.stdout.log", "/root/sample-app/log/puma.stderr.log", true

# Enable collection and sending of StatsD metrics 
plugin :statsd    

• application_controller.rb: The following code example shows the appropriate log format in the application_controller.rb file to support CloudWatch dashboard queries. Additionally, the data points and schema in this file must be consistent with PumaLogWidgets in the widget export file.

  

  application_controller.rb

• production.rb: The following code example shows how to configure the log format by commenting out request id tags and prepending the logs. This turns on full support for the CloudWatch dashboard queries.

  # Prepend all log lines with the following tags.
  # comment out line 55 in order to populate desired log metrics on CloudWatch dashboard
  # config.log_tags = [ :request_id ]  

• index.html.erb: Default home page for the Puma web server.

• routes.rb: Directs to the index page of the sample Rails application preinstalled on the Puma demo instance.

• statsd_instrument.rb: Initial configuration for statsd module.

• page_controller.rb: Uses statsd instrument gem to add any additional statsd supported metrics

• supervisord.conf: Supervisor helps manage statsd processes and auto restart statsd processes in case of failure