Amazon DocumentDB
Developer Guide

The AWS Documentation website is getting a new look!
Try it now and let us know what you think. Switch to the new look >>

You can return to the original look by selecting English in the language selector above.

Profiling Amazon DocumentDB Operations

With Amazon DocumentDB (with MongoDB compatibility), you can use the profiler to log the execution time and details of operations that were performed on your cluster. Profiler is useful for monitoring the slowest operations on your cluster to help you improve individual query performance and overall cluster performance.

By default, the profiler feature is disabled. When enabled, the profiler logs operations that are taking longer than a customer-sedefinedt threshold (e.g., 100ms) value to Amazon CloudWatch Logs. Logged details include the profiled command, time, plan summary, and client metadata. Once the operations are logged to CloudWatch Logs, you can use CloudWatch Insight to analyze, monitor, and archive your Amazon DocumentDB profiling data. Common queries are provided in the section Common Queries.

When enabled, profiler utilizes additional resources in your cluster. We recommend that you start with a high threshold value (i.e., 500ms) and gradually lower the value to identify slow operations. Starting with a threshold value of 50ms can cause performance issues on your cluster for high throughput applications. Profiler is enabled at the cluster level and is performed on all instances and databases in a cluster. Amazon DocumentDB logs operations to Amazon CloudWatch Logs on a best effort basis.

Although Amazon DocumentDB imposes no additional charge to enable the profiler, you are charged the standard rates for the usage of CloudWatch Logs. For information about CloudWatch Logs pricing, see Amazon CloudWatch pricing.

Supported Operations

Amazon DocumentDB profiler supports the following operations:

  • aggregate

  • count

  • delete

  • distinct

  • find (OP_QUERY and command)

  • findAndModify

  • insert

  • update

Limitations

The profiler does not currently support the logging of find operators that return a cursor.

Enabling Amazon DocumentDB Profiler

Enabling the profiler on a cluster is a three-step process. Ensure that all steps are completed, or profiling logs will not be sent to CloudWatch Logs. Profiler is set at the cluster level and is performed on all of the cluster's databases and instances.

To enable the profiler on a cluster

  1. Because you can't modify a default cluster parameter group, ensure that you have an available custom cluster parameter group. If you don't have a custom cluster parameter group available, then you need to create one. For more information, see Creating an Amazon DocumentDB Cluster Parameter Group

  2. Using an available custom cluster parameter group, modify the following parameters: profiler, profiler_threshold_ms, and profiler_sampling_rate.

  3. Create or modify your cluster to use the custom cluster parameter group and to enable exporting profiler logs to CloudWatch Logs.

The following sections show how to implement these steps using the AWS Management Console and the AWS CLI.

Using the AWS Management Console

  1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at https://console.aws.amazon.com/docdb.

  2. Create a custom cluster parameter group if you don't already have one. For more information, see Creating an Amazon DocumentDB Cluster Parameter Group.

  3. Using an available custom cluster parameter group, modify the following parameters. For more information, see Modifying an Amazon DocumentDB Cluster Parameter Group.

    • profiler—Enables or disables query profiling. Permitted values are enabled and disabled. The default value is disabled. To enable profiling, set the value to enabled.

    • profiler_threshold_ms—When profiler is set to enabled, all commands that are taking longer than profiler-threshold-ms are logged to CloudWatch. Permitted values are [50-INT_MAX]. Setting this value to 0 profiles all supported operations. The default value is 100.

    • profiler_sampling_rate—The fraction of slow operations that should be profiled or logged. Permitted values are [0.0-1.0].

  4. Create or modify your cluster to use the custom cluster parameter group and set the --enable-cloudwatch-logs-exports to profiler. For more information, see the following:

Using the AWS CLI

The following procedure enables the profiler on all supported operations for the cluster sample-cluster.

  1. To ensure that you have an available custom cluster parameter group, run the following command. Review the output for a cluster parameter group that doesn't have default in the name and has docdb3.6 as the parameter group family.

    aws docdb describe-db-cluster-parameter-groups \ --query 'DBClusterParameterGroups[*].[DBClusterParameterGroupName,DBParameterGroupFamily]'

    In the following output (in JSON format), only custom-pg-profiler meets both criteria.

    [ [ "custom-pg-profiler", "docdb3.6" ], [ "default.aurora5.6", "aurora5.6" ], [ "default.docdb3.6", "docdb3.6" ] ]

    If needed, create a custom cluster parameter group.

    aws docdb create-db-cluster-parameter-group \ --db-cluster-parameter-group-name custom-pg-profiler \ --db-parameter-group-family docdb3.6 \ --description "Custom cluster parameter group with profiler enabled"

    Output from this operation looks something like the following (JSON format).

    { "DBClusterParameterGroup": { "DBClusterParameterGroupName": "custom-pg-profiler", "DBParameterGroupFamily": "docdb3.6", "Description": "Custom cluster parameter group with profiler enabled", "DBClusterParameterGroupArn": "arn:aws:rds:us-east-1:111122223333:cluster-pg:custom-pg-profiler" } }
  2. Using your custom cluster parameter group, modify the following parameters:

    • profiler—Enables or disables query profiling. Permitted values are enabled and disabled. The default value is disabled. To enable profiling, set the value to enabled.

    • profiler_threshold_ms—When profiler is set to enabled, all commands taking longer than profiler-threshold-ms are logged to CloudWatch. Permitted values are [0-INT_MAX]. Setting this value to 0 profiles all supported operations. The default value is 100.

    • profiler_sampling_rate—The fraction of slow operations that should be profiled or logged. Permitted values are [0.0-1.0].

    aws docdb modify-db-cluster-parameter-group \ --db-cluster-parameter-group-name custom-pg-profiler \ --parameters ParameterName=profiler,Value=enabled,ApplyType=immediate \ ParameterName=profiler_threshols_ms,Value=100,ApplyType=immediate \ ParameterName=profiler_sampling_rate,Value=0.5,ApplyType=immediate
  3. Create or modify a cluster so that it both uses the custom-pg-profiler custom cluster parameter group and sets the parameter --enable-cloudwarch-logs-exports to profiler.

    • To create a cluster

      The following code creates the cluster sample-cluster that uses the custom-pg-profiler custom cluster parameter group from the previous step, and adds profiler to the enabled CloudWatch Logs exports.

      aws docdb create-db-cluster \ --db-cluster-identifier sample-cluster \ --engine docdb \ --master-username master-username \ --master-user-password password \ --db-subnet-group-name default \ --db-cluster-parameter-group-name custom-pg-profiler \ --enable-cloudwatch-logs-exports profiler
    • To modify a cluster

      The following code modifies the cluster sample-cluster to use the custom-pg-profiler custom cluster parameter group from the previous step, and adds profiler to the enabled CloudWatch Logs exports.

      aws docdb modify-db-cluster \ --db-cluster-identifier sample-cluster \ --db-cluster-parameter-group-name custom-pg-profiler \ --cloudwatch-logs-export-configuration '{"EnableLogTypes":["profiler"]}'

Disabling the Amazon DocumentDB Profiler

To disable the profiler, you disable both the profiler parameter and the export of profiler logs to CloudWatch Logs.

Disabling profiler

You can disable the profiler parameter using either the AWS Management Console or AWS CLI.

Using the AWS Management Console

The following procedure uses the AWS Management Console to disable Amazon DocumentDB profiler.

  1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at https://console.aws.amazon.com/docdb.

  2. In the navigation pane, choose Parameter groups. Then choose the name of the cluster parameter group that you want to disable the profiler on.

  3. On the Actions menu, choose Modify.

  4. Locate the profiler parameter. Choose the button to the left of its name, and then choose Edit.

  5. In the Modify profiler dialog box, choose disabled in the list.

  6. Choose Modify cluster parameter.

Using the AWS CLI

To disable profiler on a cluster using the AWS CLI, modify the cluster as shown here.

aws docdb modify-db-cluster-parameter-group \ --db-cluster-parameter-group-name custom-pg-profiler \ --parameters ParameterName=profiler,Value=disabled,ApplyType=immediate

Disabling profiler Logs Export

You can disable exporting profiler logs to CloudWatch Logs by using either the AWS Management Console or AWS CLI.

Using the AWS Management Console

The following procedure uses the AWS Management Console to disable Amazon DocumentDB exporting logs to CloudWatch.

  1. Open the Amazon DocumentDB console at https://console.aws.amazon.com/docdb.

  2. In the navigation pane, choose Clusters. Choose the button to the left of the name of the cluster for which you want to disable exporting logs.

  3. On the Actions menu, choose Modify.

  4. Scroll down to the Log exports section, and choose Disabled.

  5. Choose Continue.

  6. Review your changes, and then choose when you want this change applied to your cluster:

    • Apply during the next scheduled maintenance window

    • Apply immediately

  7. Choose Modify cluster.

Using the AWS CLI

The following code modifies the cluster sample-cluster and disables CloudWatch audit logs.

For Linux, macOS, or Unix:

aws docdb modify-db-cluster \ --db-cluster-identifier sample-cluster \ --cloudwatch-logs-export-configuration '{"DisableLogTypes":["profiler"]}'

For Windows:

aws docdb modify-db-cluster ^ --db-cluster-identifier sample-cluster ^ --cloudwatch-logs-export-configuration '{"DisableLogTypes":["profiler"]}'

Output from this operation looks something like the following (JSON format).

{ "DBCluster": { "DBClusterParameterGroup": "default.docdb3.6", "HostedZoneId": "ZNKXH85TT8WVW", "MasterUsername": "<user-name>", "Status": "available", "Engine": "docdb", "Port": 27017, "AvailabilityZones": [ "us-east-1a", "us-east-1c", "us-east-1f" ], "EarliestRestorableTime": "2019-02-13T16:35:50.387Z", "DBSubnetGroup": "default", "LatestRestorableTime": "2019-02-13T16:35:50.387Z", "DBClusterArn": "arn:aws:rds:us-east-1:111122223333:cluster:sample-cluster2", "Endpoint": "sample-cluster2.cluster-corcjozrlsfc.us-east-1.docdb.amazonaws.com", "ReaderEndpoint": "sample-cluster2.cluster-ro-corcjozrlsfc.us-east-1.docdb.amazonaws.com", "BackupRetentionPeriod": 1, "EngineVersion": "3.6.0", "MultiAZ": false, "ClusterCreateTime": "2019-02-13T16:35:04.756Z", "DBClusterIdentifier": "sample-cluster2", "AssociatedRoles": [], "PreferredBackupWindow": "07:16-07:46", "DbClusterResourceId": "cluster-YOS52CUXGDTNKDQ7DH72I4LED4", "StorageEncrypted": false, "PreferredMaintenanceWindow": "wed:03:08-wed:03:38", "DBClusterMembers": [], "VpcSecurityGroups": [ { "Status": "active", "VpcSecurityGroupId": "sg-77186e0d" } ] } }

Accessing Your Amazon DocumentDB Profiler Logs

Use the following steps to access your profile logs on Amazon CloudWatch.

  1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

  2. Make sure that you are in the same Region as your Amazon DocumentDB cluster.

  3. In the navigation pane, choose Logs.

  4. To find the profile logs for your cluster, in the list, choose /aws/docdb/yourClusterName/profile.

    The profile logs for each of your instances are available under each of the respective instance names.

Common Queries

Following are some common queries you can use to analyze your profiled commands. For more information on CloudWatch Insights, see Analyzing Log Data with CloudWatch Logs Insights and sample queries.

Get 10 slowest operations on a specified collection

filter ns="test.foo" | sort millis desc | limit 10

Get all the update operations on a collection which took more than 20 ms

filter millis > 20 and op = "update"

Get 10 slowest operations in last month

sort millis desc | limit 10

Get all the queries with a COLLSCAN plan summary

filter planSummary="COLLSCAN"