Transformations - Amazon Managed Grafana
Applying a transformationTransformation types and options

Transformations

This documentation topic is designed for Grafana workspaces that support Grafana version 8.x.

For Grafana workspaces that support Grafana version 10.x, see Working in Grafana version 10.

For Grafana workspaces that support Grafana version 9.x, see Working in Grafana version 9.

Transformations process the result set before it’s passed to the visualization. You access transformations in the Transform tab of the Amazon Managed Grafana panel editor.

You can use transformations to rename fields, join separate time series together, do math across queries, and more. If you have large dashboards or heavy queries, being able to reuse the query result from one panel to another panel can provide a huge performance gain.

Note

Transformations sometimes result in data that cannot be graphed. When that happens, Amazon Managed Grafana displays a suggestion on the visualization. Choose the suggestion to switch to table visualization. This often helps you better understand what the transformation is doing to your data.

Amazon Managed Grafana applies transformations in the sequence that they are listed on the screen. Every transformation creates a new result set that is passed to the next transformation in the pipeline.

The order can make a huge difference in how your results look. For example, if you use reduce transformation to condense all the results of one column to a single value, you can apply transformations only to that single value.

Prerequisites

Before you apply transformations, all of the following must be true:

  • You have entered a query and returned data from a data source. For more information about queries, see Queries.

  • You have applied a visualization that supports queries, such as one of the following visualizations:

    • Bar gauge

    • Gauge

    • Graph

    • Heatmap

    • Logs

    • Stat

    • Table

Applying a transformation

Transformations are available from the Transform tab in the bottom pane of the panel editor, next to the Queries tab.

To apply a transformation

  1. On the panel that you want to add transformations to, choose the panel title, and then choose Edit.

  2. Choose the Transform tab.

  3. Select a transformation.

    In the transformation row that appears, you can configure the transformation options.

  4. To apply another transformation, choose Add transformation. Keep in mind that the next transformation acts on the result set returned by the previous transformation.

If you have trouble, choose the bug icon to debug your transformations.

To remove a transformation, choose the trash can icon.

Transformation types and options

Grafana workspaces include the following transformations.

Reduce

Apply a Reduce transformation when you want to simplify your results down to one value. Reduce basically removes the time component. If visualized as a table, it reduces a column down to one row (value).

In the Calculations field, enter one or more calculation types. Choose to see a list of calculation choices. For information about available calculations, see Calculations list.

After you select at least one calculation, Amazon Managed Grafana displays one value using the calculation you selected. If you select more than one calculation, more than one value is displayed.

Merge

Use this transformation to combine the results from multiple queries into one single result. This is helpful when using the table panel visualization. Values that can be merged are combined into the same row. Values can be merged if the shared fields contain the same data.

In the following example, two queries return table data. The data are visualized as two separate tables before applying the transformation.

Query A

Time Job Uptime
2020-07-07 11:34:20 node 25260122
2020-07-07 11:24:20 postgre 123001233

Query B

Time Job Errors
2020-07-07 11:34:20 node 15
2020-07-07 11:24:20 postgre 5

Here is the result after applying the Merge transformation.

Time Job Errors Uptime
2020-07-07 11:34:20 node 15 25260122
2020-07-07 11:24:20 postgre 5 123001233

Filter by name

Use this transformation to remove portions of the query results.

Amazon Managed Grafana displays the Identifier field, followed by the fields returned by your query.

You can apply filters in one of two ways:

  • Enter a regex expression.

  • Choose a field to toggle filtering on that field. Filtered fields are displayed with dark gray text, unfiltered fields have white text.

Filter data by query

Use this transformation in panels that have multiple queries, if you want to hide one or more of the queries.

Amazon Managed Grafana displays the query identification letters in dark gray text. To toggle filtering, choose a query identifier. If the query letter is white, the results are displayed. If the query letter is dark, the results are hidden.

Organize fields

Use this transformation to rename, reorder, or hide fields returned by the query.

Note

This transformation works only in panels that have a single query. If your panel has multiple queries, you must either apply a Join by field (outer join) transformation or remove the extra queries.

Amazon Managed Grafana displays a list of fields returned by the query. You can make any of the following changes:

  • Change the field order by pausing over a field. The cursor turns into a hand, and then you can drag the field to its new place.

  • Hide or show a field by choosing the eye icon next to the field name.

  • Rename fields by typing a new name in the Rename box.

Join by field (outer join)

Use this transformation to join multiple time series from a result set by field.

This transformation is useful if you want to combine queries so that you can calculate results from the fields.

Add field from calculation

Use this transformation to add a new field calculated from two other fields. Each transformation allows you to add one new field.

  • Mode – Select a mode:

    • Reduce row – Apply selected calculation on each row of selected fields independently.

    • Binary option – Apply a basic math operation (sum, multiply, and so on) on values in a single row from two selected fields.

  • Field name – Select the names of fields that you want to use in the calculation for the new field.

  • Calculation - Select a calculation to use when Amazon Managed Grafana creates the new field. Choose the field to see a list of calculation choices. For information about available calculations, see Calculations list.

  • Alias – (Optional) Enter the name of your new field. If you leave this blank, the field will be named to match the calculation.

  • Replace all fields – (Optional) Use this option if you want to hide all other fields and display only your calculated field in the visualization.

Labels to fields

Note

To apply this transformation, your query needs to return labeled fields.

When you select this transformation, Amazon Managed Grafana automatically transforms all labeled data into fields.

For example, consider a query result of two time series.

1: labels Server=Server A, Datacenter=EU 2: labels Server=Server B, Datacenter=EU

This transformation would result in the following table.

Time Server Datacenter Value
2020-07-07 11:34:20 Server A EU 1
2020-07-07 11:34:20 Server B EU 2

Value field name; If you selected Server as the Value field name, you would get one field for every value of the Server label.

Time Datacenter Server A Server B
2020-07-07 11:34:20 EU 1 2

Group By

This transformation sorts each frame by the configured field. When reverse is checked, the values are returned in the opposite order.

Group By

This transformation groups the data by a specified field (column) value and processes calculations on each group. The available calculations are the same as for the Reduce transformation.

Here’s an example of original data.

Time Server ID CPU Temperature Server Status
2020-07-07 11:34:20 server 1 80 Shutdown
2020-07-07 11:34:20 server 3 62 OK
2020-07-07 10:32:20 server 2 90 Overload
2020-07-07 10:31:22 server 3 55 OK
2020-07-07 09:30:57 server 3 62 Rebooting
2020-07-07 09:30:05 server 2 88 OK
2020-07-07 09:28:06 server 1 80 OK
2020-07-07 09:25:05 server 2 88 OK
2020-07-07 09:23:07 server 1 86 OK

This transformation takes two steps. First, you specify one or multiple fields to group the data by. This will group all the same values of those fields together, as if you sorted them. For instance, if you Group By the Server ID field, it will group the data this way:

Time Server ID CPU Temperature Server Status
2020-07-07 11:34:20 server 1 80 Shutdown
2020-07-07 09:28:06 server 1 80 OK
2020-07-07 09:23:07 server 1 86 OK 
2020-07-07 10:32:20 | server 2 | 90 | Overload
2020-07-07 09:30:05 | server 2 | 88 | OK
2020-07-07 09:25:05 | server 2 | 88 | OK

2020-07-07 11:34:20 | server 3 | 62 | OK
2020-07-07 10:31:22 | server 3 | 55 | OK
2020-07-07 09:30:57 | server 3 | 62 | Rebooting

All rows with the same value of Server ID are grouped together.

After choosing which field you want to group your data by, you can add various calculations on the other fields, and the calculation will be applied on each group of rows. For instance, you might want to calculate the average CPU temperature for each of those servers. You can add the mean calculation applied on the CPU Temperature field to get the following.

Server ID CPU Temperature (mean)
server 1 82
server 2 88.6
server 3 59.6

And you can add more than one of those calculations. For instance, you can use the following calculations.

  • For field Time, you can calculate the Last value, to know when the last data point was received for each server.

  • For field Server Status, you can calculate the Last value to know the last state value for each server.

  • For field Temperature, you can also calculate the Last value to know the latest monitored temperature for each server.

The Group By transformation produces the following results.

Server ID CPU Temperature (mean) CPU Temperature (last) Time (last) Server Status (last)
server 1 82 80 2020-07-07 11:34:20 Shutdown
server 2 88.6 90 2020-07-07 10:32:20 Overload
server 3 59.6 62 2020-07-07 11:34:20 OK

Using this transformation, you can extract some key information out of your time series and display it in a convenient way.

Series to rows

Use this transformation to combine the results from multiple time series data queries into one single result. This is helpful when using the table panel visualization.

The result from this transformation will contain three columns: Time, Metric, and Value. The Metric column is added so that you can see which query the metric originates from. Customize this value by defining Label on the source query.

In the example below, two queries return time series data. It is visualized as two separate tables before the transformation is applied.

Query A

Time Temperature
2020-07-07 11:34:20 25
2020-07-07 10:31:22 22
2020-07-07 09:30:05 19

Query B

Time Humidity
2020-07-07 11:34:20 24
2020-07-07 10:32:20 29
2020-07-07 09:30:57 33

Applying the Series to rows transformation produces the following results.

Time Metric Value
2020-07-07 11:34:20 Temperature 25
2020-07-07 11:34:20 Humidity 22
2020-07-07 10:32:20 Humidity 29
2020-07-07 10:31:22 Temperature 22
2020-07-07 09:30:57 Humidity 33
2020-07-07 09:30:05 Temperature 19

Filter data by value

This transformation allows you to filter your data directly in the Grafana workspace and remove some data points from your query result. You have the option to include or exclude data that match one or more conditions you define. The conditions are applied on a selected field.

This transformation is useful if your data source does not natively filter by values. You might also use this to narrow values to display if you are using a shared query.

The available conditions for all fields are as follows:

  • Regex – Match a regex expression.

  • Is Null – Match if the value is null.

  • Is Not Null – Match if the value is not null.

  • Equal – Match if the value is equal to the specified value.

  • Different – Match if the value is different than the specified value.

The available conditions for number fields are as follows:

  • Greater – Match if the value is greater than the specified value.

  • Lower – Match if the value is lower than the specified value.

  • Greater or equal – Match if the value is greater than or equal to the specified value.

  • Lower or equal – Match if the value is lower than or equal to the specified value.

  • Range – Match a range between a specified minimum and maximum. The minimum and maximum are included in the range.

You can add more than one condition to the filter. When you have more than one condition, you can choose if you want the include or exclude action to be applied on rows that Match all or any of the conditions you added.

Conditions that are not valid or incompletely configured are ignored.

Debug transformations

To see the input and the output result sets of the transformation, choose the bug icon on the right side of the transformation row.

Amazon Managed Grafana displays the transformation debug view below the transformation row.