Nginx - Centralized Logging with OpenSearch

Nginx

Nginx is capable of writing error and access log files to a local directory. You can configure Centralized Logging with OpenSearch to ingest Nginx logs.

Prerequisites

Make sure you have done the following:

Step 1: Create a Nginx log config

  1. Sign in to the Centralized Logging with OpenSearch Console.

  2. In the left sidebar, under Resources, choose Log Config.

  3. Click the Create a log config button.

  4. Specify Config Name.

  5. Specify Log Path. You can use , to separate multiple paths.

  6. Choose Nginx in the log type dropdown menu.

  7. In the Nginx Log Format section, paste your Nginx log format configuration. It is in the format of /etc/nginx/nginx.conf and starts with log_format.

    For example:

    log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"';
  8. (Optional) In the Sample log parsing section, paste a sample Nginx log to verify if the log parsing is successful.

    For example:

    127.0.0.1 - - [24/Dec/2021:01:27:11 +0000] "GET / HTTP/1.1" 200 3520 "-" "curl/7.79.1" "-"
  9. (Optional) In the Filter section, you add some conditions to filter logs at the log agent side. The solution will ingest logs that match ALL the specified conditions only.

  10. Select Create.

Step 2: Create an application log ingestion

Instance Group as Log Source

  1. Sign in to the Centralized Logging with OpenSearch Console.

  2. In the left sidebar, under Log Analytics Pipelines, choose Application Log.

  3. Choose the application pipeline that has been created during the Prerequisites.

  4. Choose the Permission grant method. If you choose I will manually add the below required permissions after pipeline creation, you have to click Expand to view required permissions and copy the provided JSON policy.

  5. Go to AWS Management Console > IAM > Policies on the left column, and

    1. Choose Create Policy, choose JSON and replace all the content inside the text block. Remember to substitute <YOUR ACCOUNT ID> with your account id.

    2. Choose Next, Next, then enter the name for this policy.

    3. Attach the policy to your EC2 instance profile to grant the log agent permissions to send logs to the application log pipeline. If you are using Auto Scaling group, you need to update the IAM instance profile associated with the Auto Scaling Group. If needed, you can follow the documentation to update your launch template or launch configuration.

  6. Click the Create an Ingestion dropdown menu, and select From Instance Group.

  7. Select Choose exists and choose Next.

  8. Select the instance group you have created during the Prerequisites and choose Next.

  9. (Auto Scaling Group only) If your instance group is created based on an Auto Scaling Group, after ingestion status become "Created", then you can find the generated Shell Script in the instance group's detail page. Copy the shell script and update the User Data of the Auto Scaling Launch configurations or Launch template.

  10. Select Choose exists and select the log config created in previous setup.

  11. Choose Create to finish creating an ingestion.

Amazon EKS Cluster as Log Source

  1. Sign in to the Centralized Logging with OpenSearch Console.

  2. In the left sidebar, under Log Source, choose EKS Clusters.

  3. Choose the EKS Cluster that has been imported as Log Source during the Prerequisites.

  4. Go to App Log Ingestion tab and choose Create an Ingestion.

    1. Select Choose exists and choose the application pipeline created during the Prerequisites. Choose Next.

    2. Select the log config created in previous setup.

    3. Choose Create to finish creating an ingestion.

  5. Deploy Fluent-bit log agent following the guide generated by Centralized Logging with OpenSearch.

    1. Select the App Log Ingestion just created.

    2. Follow DaemonSet or Sidecar Guide to deploy the log agent.

Step 3: Check built-in Nginx dashboard in OpenSearch

For Nginx logs, Centralized Logging with OpenSearch creates a built-in sample dashboard.

  1. Open OpenSearch dashboard in your browser.

  2. Go to Dashboard section in the left sidebar.

  3. Find the dashboard whose name starts with <the application pipeline>.

View dashboard

The dashboard includes the following visualizations.

Visualization Name Source Field Description
Total Requests
  • log event

Displays aggregated events based on a specified time interval.
Http Method
  • request_method

Presents a pipe chart that shows the distribution of request methods handled by Nginx during a selected time period.
Request History
  • log event

Shows a historical log of all requests handled by Nginx, visualized using a bar chart. This allows administrators to analyze traffic volumes and patterns over time.
Unique Visitors
  • remote_addr

Shows the number of distinct IP addresses that have made requests to the application over a given time period.
Bandwidth
  • body_bytes_sent

The bandwidth metric tracks the total amount of data transferred to clients by the Nginx server over time.
Status Code Metric
  • status

Displays the distribution of HTTP response codes served by the Nginx server over a period of time.
Status Code
  • status

The proportion of each status code relative to the total number of responses is also displayed as a percentage. This allows easy identification of the dominant response types.
Bandwidth History
  • body_bytes_sent

Shows the historical trend of the data transfer activities by the Nginx server to clients.
Top IPs
  • body_bytes_sent

  • remote_addr

Displays the 10 client IP addresses generating the most requests to the application during a specified time period.
Top Referers
  • http_referer

Referers are the URLs of pages that link to requests for the application. Tracking referers reveal the primary external sources of visits and engagement.
Top User Agents
  • http_user_agent

Shows the breakdown of client browser and device types generating traffic.
Top Access URL
  • remote

  • method

Shows the most frequently requested URLs on the application during a specified time period.
Nginx Error Log
  • @timestamp

  • status

  • remote_addr

  • request_uri

  • request_method

  • http_referer

  • body_bytes_sent

  • http_user_agent

Provides a detailed record of errors encountered by the web server.

Sample Dashboard

You can access the built-in dashboard in Amazon OpenSearch Service to view log data. For more information, see Access Dashboard.

You can click the below image to view the high-resolution sample dashboard.