CloudWatch Logs Insights Query Syntax
CloudWatch Logs Insights supports a query language you can use to perform queries
on your log groups. Each query can include one or more query commands separated by
Unix-style pipe characters (|
).
Six query commands are supported, along with many supporting functions and operations, including regular expressions, arithmetic operations, comparison operations, numeric functions, datetime functions, string functions, and generic functions.
Comments are also supported. Lines in a query that start with the #
character are ignored.
Fields that start with the @
symbol are generated by CloudWatch Logs Insights. For more
information about the fields that CloudWatch Logs discovers automatically and generates,
see
Supported Logs and Discovered
Fields.
CloudWatch Logs Insights Query Commands
The following table lists the six supported query commands along with basic examples. For more powerful sample queries, see Sample Queries.
Command | Description | Examples |
---|---|---|
|
Specifies which fields to display in the query results. If you specify this command more than once in your query, only the fields you specify in the last occurrence are used. |
The following example uses the field
|
|
Retrieves the specified fields from log events for display. You can use functions and operations within a fields command to modify field values for display and to create new fields for use in the rest of the query. |
The following example displays the fields
The following example creates and displays an ephemeral field
|
|
Filters the results of a query based on one or more conditions. You can use
a variety of operators and expressions in the |
The following example retrieves the fields
The following example also a valid query, but the
results don't display separate fields. Instead, the results display
the
The following example retrieves the fields
The next example returns log
events where the field
The next example returns log events that have a
This final example returns log events that do not
have
|
|
Calculates aggregate statistics based on the values of log fields.
When you use Several statistical operators are supported, including
|
The following example calculates the average value of
|
|
Sorts the retrieved log events. Both ascending ( |
The following example sorts the returned events in descending order based on the value
of
|
|
Specifies the number of log events returned by the query. You can use this to limit the results to a small number to see a small set of relevant
results. You can also use
If you don't specify a limit, the query defaults to displaying a maximum of 1000 rows. |
The following example sorts the events in descending
order based on the value of
|
|
Extracts data from a log field and creates one or more ephemeral fields that you can
process further in the query.
For glob expressions, provide the Enclose regular expressions in forward slashes (/). Within the
expression, each part of the matched string that is to be
extracted is enclosed in a named capturing group. An example of
a named capturing group is |
Using this single log line as an example:
The following two
The following example uses a regular expression to extract the ephemeral fields
|
Notes about query commands in the previous table
The following rules, guidelines, and tips apply to the query commands in the previous table.
-
Any log field named in a query that has characters other than the @ sign, the period (.), and alphanumeric characters must be surrounded by backtick (
`
) characters. For example, thefoo-bar
field name must be enclosed in backtick characters because it includes a non-alphanumeric character. -
Both
fields
anddisplay
are used to specify the fields to display in the query results. The differences between the two are as follows:-
You use the
display
command only to specify which fields to display in the results. You can use thefields
command with the as keyword to create new ephemeral fields using functions and the fields that are in the log event. For example,fields ispresent(resolverArn) as isRes
creates an ephemeral field namedisRes
which can be used in the remainder of the query. The value ofisRes
is either 0 or 1 depending on whether or notresolverArn
is a discovered field in the log event. -
If you have multiple
fields
commands and do not include adisplay
command, the fields specified in all of thefields commands are displayed.
-
If you have multiple
display
commands, only the fields specified in the finaldisplay command are displayed.
-
Matches and Regular Expressions in the Filter Command
You can use
comparison operators (=, !=, <, <=, >, >=), Boolean
operators (and
, or
, and
not
) and regular expressions in the
filter
command.
You can use in
to test for set membership. Put an array with the elements
to check for immediately after in
. You can use not
with in
. String matches using in
must be complete string matches.
To filter by substrings, you can use like
or =~
(equal sign followed by a tilde) in the
filter
command. For a substring match using like
or =~
, enclose your substring to match with double or
single quotation marks. To perform regular expression
matching, enclose the expression to match with forward slashes. The query returns
only log events that
match the criteria that you set.
Examples
The following three examples return all events in which f1
contains
the word Exception
. The first two examples use regular expressions. The
third example uses a substring match. All three examples are case sensitive.
fields f1, f2, f3 | filter f1 like /Exception/
fields f1, f2, f3 | filter f1 =~ /Exception/
fields f1, f2, f3 | filter f1 like "Exception"
The following example changes the search for "Exception" to not be case sensitive.
fields f1, f2, f3 | filter f1 like /(?i)Exception/
The following example uses a regular expression. It returns all events in which
f1
is exactly the word Exception
. The query isn't case
sensitive.
fields f1, f2, f3 | filter f1 =~ /^(?i)Exception$/
Using Aliases in Queries
You can use as
to create one or more aliases in a query. Aliases are
supported in the fields
,
stats
, and sort
commands.
You can create aliases for log fields and for the results of operations and functions.
Examples
The following examples show the use of aliases in query commands.
fields abs(myField) as AbsoluteValuemyField, myField2
Returns the absolute value of myField
as
AbsoluteValuemyField
and also returns the field
myField2
.
stats avg(f1) as myAvgF1 | sort myAvgF1 desc
Calculates the average of the values of the f1
as myAvgF1
and returns them in descending order by that value.
Using Comments in Queries
You can comment out lines in a query by using the #
character. Lines that start with the #
character are ignored.
This can be useful to document your query or to temporarily ignore part of a complex
query for one call, without deleting that line.
In the following example, the second line of the query is ignored.
fields @timestamp, @message # | filter @message like /delay/ | limit 20
Supported Operations and Functions
The query language supports many types of operations and functions, as shown in the following tables.
Comparison Operations
You can use comparison operations in the filter
command and as arguments for other functions. Comparison operations
accept all data types as arguments and return a Boolean result.
= != < <= > >=
Boolean Operators
You can use the Boolean operators and
, or
, and
not
. You can use these Boolean
operators only in functions that return a Boolean value.
Arithmetic Operations
You can use arithmetic operations in the filter
and fields
commands and as arguments for other functions.
Arithmetic operations
accept numeric data types as arguments and return numeric results.
Operation | Description |
---|---|
|
Addition |
|
Subtraction |
|
Multiplication |
|
Division |
|
Exponentiation. |
|
Remainder or modulus. |
Numeric Operations
You can use numeric operations in the filter
and fields
commands and as arguments for other functions.
Numeric operations accept numeric data types as arguments and return numeric results.
Operation | Result Type | Description |
---|---|---|
|
number |
Absolute value. |
|
number |
Round to ceiling (the smallest integer that is
greater than the value of |
|
number |
Round to floor (the largest integer that is
smaller than the value of |
|
number |
Returns the largest value. |
|
number |
Returns the smallest value. |
|
number |
Natural log. |
|
number |
Square root. |
General Functions
You can use general functions in the filter
and fields
commands and as arguments for other functions.
Function | Result Type | Description |
---|---|---|
|
boolean |
Returns |
|
LogField |
Returns the first non-null value from the list. |
String Functions
You can use string functions in the filter
and fields
commands and as arguments for other functions.
Function | Result Type | Description |
---|---|---|
|
boolean |
Returns |
|
boolean |
Returns |
|
string |
Concatenates the strings. |
|
string |
Removes white space from the left of the string. If the
function has a second string argument, it removes the characters
of |
|
string |
Removes white space from the right of the string. If the
function has a second string argument, it removes the characters
of |
|
string |
Removes white space from both ends of the string. If the
function has a second string argument, it removes the characters
of |
|
number |
Returns the length of the string in Unicode code points. |
|
string |
Converts the string to uppercase. |
|
string |
Converts the string to lowercase. |
|
string |
Returns a substring from the index specified by the number argument to the end of
the string. If the function has a second number argument, it contains the length of
the
substring to be retrieved. For example, |
|
string |
Replaces all instances of |
|
number |
Returns 1 if |
Datetime Functions
You can use datetime functions in the filter
and fields
commands and as arguments for other functions.
You can use these functions to create time buckets for queries with aggregate functions.
As part of datetime functions, you can use time periods that consist of a number
and then either m
for minutes or h
for hours. For example,
10m
is 10 minutes and 1h
is 1 hour.
Function | Result Type | Description |
---|---|---|
|
Timestamp |
Rounds the value of |
|
Timestamp |
Truncates the timestamp to the given period. For example, |
|
Timestamp |
Rounds up the timestamp to the given period and then truncates. For example, |
|
Timestamp |
Interprets the input field as the number of milliseconds since the Unix epoch and converts it to a timestamp. |
|
number |
Converts the timestamp found in the named field into a number representing the milliseconds since the Unix epoch. |
IP Address Functions
You can use IP address string functions in the filter
and fields
commands and as arguments for other functions.
Function | Result Type | Description |
---|---|---|
|
boolean |
Returns |
|
boolean |
Returns |
|
boolean |
Returns |
|
boolean |
Returns |
|
boolean |
Returns |
|
boolean |
Returns |
Stats Aggregation Functions
You can use aggregation functions in the stats
command and as arguments for other functions.
Function | Result Type | Description |
---|---|---|
|
number |
The average of the values in the specified field. |
|
number |
Counts the log events. |
|
number |
Returns the number of unique values for the field. If the field has very high cardinality
(contains many unique values), the value returned by |
|
LogFieldValue |
The maximum of the values for this log field in the queried logs. |
|
LogFieldValue |
The minimum of the values for this log field in the queried logs. |
|
LogFieldValue |
A percentile indicates the relative standing of
a value in a dataset. For example,
|
|
number |
The standard deviation of the values in the specified field. |
|
number |
The sum of the values in the specified field. |
Stats Non-Aggregation Functions
You can use non-aggregation functions in the stats
command and as arguments for other functions.
Function | Result type | Description |
---|---|---|
|
LogField |
Returns the value of |
|
LogField |
Returns the value of |
|
LogField |
Returns the value of |
|
LogField |
Returns the value of |