Filter pattern syntax for metric filters
Note
How metric filters differ CloudWatch Logs Insights queries
Metric filters differ from CloudWatch Logs Insights queries in that a specified numerical value is added to a metric filter each time a matching log is found. For more information, see Configuring metric values for a metric filter.
For information about how to query your log groups with the Amazon CloudWatch Logs Insights query language, see CloudWatch Logs Insights query syntax.
Generic filter pattern examples
For more information on generic filter pattern syntax applicable to metric filters as well as subscription filters and filter log events, see Filter pattern syntax for metric filters, subscription filters, and filter log events, which includes the following examples:
Supported regular expressions (regex) syntax
Matching terms in unstructured log events
Matching terms in JSON log events
Matching terms in space-delimited log events
Metric filters allow you to search and filter log data coming into CloudWatch Logs, extract metric observations from the filtered log data, and transform the data points into a CloudWatch Logs metric. You define the terms and patterns to look for in log data as it is sent to CloudWatch Logs. Metric filters are assigned to log groups, and all of the filters assigned to a log group are applied to their log streams.
When a metric filter matches a term, it increments the metric's count by a specified numerical value. For example, you can create a metric filter that counts the number of times the word ERROR occurs in your log events.
You can assign units of measure and dimensions to metrics. For example, if you create a
metric filter that counts the number of times the word
ERROR occurs in your log events, you
can specify a dimension that's called ErrorCode
to show the total number of
log events that contain the word ERROR and
filter data by reported error codes.
Tip
When you assign a unit of measure to a metric, make sure to specify the correct one. If you change the unit later, your change might not take effect. For the complete list of the units that CloudWatch supports, see MetricDatum in the Amazon CloudWatch API Reference.
Topics
Configuring metric values for a metric filter
When you create a metric filter, you define your filter pattern and specify your metric's value and default value. You can set metric values to numbers, named identifiers, or numeric identifiers. If you don't specify a default value, CloudWatch won't report data when your metric filter doesn't find a match. We recommend that you specify a default value, even if the value is 0. Setting a default value helps CloudWatch report data more accurately and prevents CloudWatch from aggregating spotty metrics. CloudWatch aggregates and reports metric values every minute.
When your metric filter finds a match in your log events, it increments your metric's count by your metric's value. If your metric filter doesn't find a match, CloudWatch reports the metric's default value. For example, your log group publishes two records every minute, the metric value is 1, and the default value is 0. If your metric filter finds matches in both log records within the first minute, the metric value for that minute is 2. If your metric filter doesn't find matches in either records during the second minute, the default value for that minute is 0. If you assign dimensions to metrics that metric filters generate, you can't specify default values for those metrics.
You also can set up a metric filter to increment a metric with a value extracted from a log event, instead of a static value. For more information, see Using values in log events to increment a metric's value.
Publishing dimensions with metrics from values in JSON or space-delimited log events
You can use the CloudWatch console or AWS CLI to create metric filters that publish dimensions with metrics that JSON and space-delimited log events generate. Dimensions are name/value value pairs and only available for JSON and space-delimited filter patterns. You can create JSON and space-delimited metric filters with up to three dimensions. For more information about dimensions and information about how to assign dimensions to metrics, see the following sections:
Dimensions in the Amazon CloudWatch User guide
Example: Extract fields from an Apache log and assign dimensions in the Amazon CloudWatch Logs User Guide
Important
Dimensions contain values
that gather charges the same as custom metrics.
To prevent unexpected charges,
don't specify high-cardinality fields,
such as IPAddress
or requestID
,
as dimensions.
If you extract metrics from log events, you're charged for custom metrics. To prevent you from collecting accidental high charges, Amazon might disable your metric filter if it generates 1000 different name/value pairs for specified dimensions over a certain amount of time.
You can create billing alarms that notify you of your estimated charges. For more information, see Creating a billing alarm to monitor your estimated AWS charges.
The following examples contain code snippets that describe how to specify dimensions in a JSON metric filter.
The following examples contain code snippets that describe how to specify dimensions in a space-delimited metric filter.
Using values in log events to increment a metric's value
You can create metric filters that publish numeric values found in your log events. The procedure in this section uses the following example metric filter to show how you can publish a numeric value in a JSON log event to a metric.
{ $.latency = * } metricValue: $.latency
To create a metric filter that publishes a value in a log event
Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/
. -
In the navigation pane, choose Logs, and then choose Log groups.
-
Select or create a log group.
For information about how to create a log group, see Create a log group in CloudWatch Logs in the Amazon CloudWatch Logs User Guide.
-
Choose Actions, and then choose Create metric filter.
-
For Filter Pattern, enter
{ $.latency = * }
, and then choose Next. -
For Metric Name, enter myMetric.
-
For Metric Value, enter
$.latency
. (Optional) For Default Value, enter 0, and then choose Next.
We recommend that you specify a default value, even if the value is 0. Setting a default value helps CloudWatch report data more accurately and prevents CloudWatch from aggregating spotty metrics. CloudWatch aggregates and reports metric values every minute.
Choose Create metric filter.
The example metric filter matches the term "latency"
in the example JSON log event
and publishes a numeric value
of 50
to the metric myMetric.
{ "latency": 50, "requestType": "GET" }