Exporting recommendations - AWS Compute Optimizer

Exporting recommendations

You can export your recommendations to record them over time, and share the data with others. Recommendations are exported in a comma-separated values (.csv) file, and its metadata in a JavaScript Object Notation (.json) file, to an existing Amazon Simple Storage Service (Amazon S3) bucket that you specify.

Restrictions

The following restrictions apply when exporting recommendations:

  • You can have only one recommendations export job in progress for each resource type, and for each AWS Region. Confirm that all previous export jobs are completed before creating a new export job. For more information about viewing your export jobs, including those that are in progress, see Viewing your export jobs.

  • Recommendations for each resource type, and in each AWS Region are exported in separate .csv files. You cannot export recommendations from multiple resource types and Regions into a single file.

  • Large export jobs can take a while to complete. Consider limiting the recommendation columns that you include in your export job. Additionally, if your account is the master account of an organization, consider limiting the number of member accounts to include in your export job.

Amazon S3 bucket permission requirements

You must create the destination S3 bucket for your recommendations export before you create the export job. Compute Optimizer does not create the S3 bucket for you. The S3 bucket that you specify for your recommendations export files cannot be publicly accessible, and cannot be configured as a Requester Pays bucket. After you create the S3 bucket, ensure that it has the required permission policy to allow Compute Optimizer to write the export files to it. If you plan to specify an object prefix when you create your recommendations export job, you must include the object prefix in the policy that you add to the S3 bucket. For more information, see Amazon S3 bucket policy for Compute Optimizer.

Exporting your recommendations

Use the following procedure to export your recommendations.

To export your recommendations

  1. Open the Compute Optimizer console at https://console.aws.amazon.com/compute-optimizer/.

  2. Choose a resource type in the navigation pane. For example, choose EC2 instances, or Auto Scaling groups.

  3. On the Recommendations page, choose the Action dropdown menu, and choose Export Recommendations.

  4. On the Export Recommendations page, under Export destination settings, specify the following:

    1. For Region, specify an AWS Region for your export.

    2. For Destination S3 bucket name, specify the name of an existing S3 bucket in the specified Region.

    3. (Optional) Choose Add Region to export the recommendations for another AWS Region.

    4. (Optional) Choose Remove next to a specified Region and S3 bucket name to remove the destination from the export job.

    5. (Optional) For Object prefix, specify a prefix to use in the destination S3 bucket for all of the export files. The prefix is an optional addition to the S3 object key that organizes your export files in your S3 bucket. For example, you can specify a date prefix (e.g., 2020/april), a resource type prefix (e.g., ec2-instances), or a combination of both (e.g., 2020/april/ec2-instances).

  5. Under Export filters, specify the following:

    1. For Resource type, choose the resource type to include in your recommendations export.

    2. For Accounts, choose if you want to include recommendations for all member accounts of the organization. This option is available only if your account is the master account of an organization.

  6. Under Columns to include, choose the recommendations data to include in your recommendations export. For more information about the columns to include, see Exported files.

  7. Choose Export after you confirm that the export job is configured correctly, or choose Cancel to return to the Recommendations page without creating the export job. The export job configuration is deleted if you cancel it.

    Note

    If you export recommendations for multiple AWS Regions at one time, they are treated as separate export jobs and Compute Optimizer will try to start all of them at once. The Export Recommendations page will display an error if an export job fails to start. Export jobs that successfully start will continue to process, but you must resolve the errors for the failed jobs before trying to start them again.

Your recommendations export job will take a while to complete. Check the status of your export jobs by viewing the Exports page. For more information, see Viewing your export jobs. Your recommendations export file, and its associated metadata file, are saved to the specified S3 bucket when the export job is completed. The following are examples of the full Amazon S3 object key for the export file and its associated metadata file. The account ID in the object keys is the account of the requester of the export job. For more information, see Exported files.

s3://<BucketName>/<OptionalPrefix>/compute-optimizer/<AccountId>/<AWSRegion>-<CreatedTimestamp>-<UniqueJobID>.csv
s3://<BucketName>/<OptionalPrefix>/compute-optimizer/<AccountId>/<AWSRegion>-<CreatedTimestamp>-<UniqueJobID>-metadata.json

Example:

s3://compute-optimizer-exports/ec2-instance-recommendations/compute-optimizer/111122223333/us-west-2-2020-03-03T133027-3e496c549301c8a4dfcsdX.csv
s3://compute-optimizer-exports/ec2-instance-recommendations/compute-optimizer/111122223333/us-west-2-2020-03-03T133027-3e496c549301c8a4dfcsdX-metadata.json

Viewing your export jobs

Use the following procedure to view the export jobs created in the last seven days.

To view your export jobs

  1. Open the Compute Optimizer console at https://console.aws.amazon.com/compute-optimizer/.

  2. Choose Exports in the navigation pane.

    The Exports page displays the recommendation export jobs that were created in the last seven days.

  3. You can perform the following actions on the page:

    • Choose the export destination link for a completed job to access the destination S3 bucket. The export destination displays only for successful export jobs. Export jobs that are in progress, or that have failed display a dash.

    • Scroll right to view the failure reason for failed export jobs. Use the failure reason to determine why your export job was not completed.

Exported files

Recommendations are exported in a.csv file, and its metadata in a .json file, to the Amazon S3 bucket that you specified when you created the export job.

Recommendations file

The recommendations file includes the recommendation data for the recommendation columns that you choose to include when you create the export job. The following table lists all of the recommendation columns that can be included in the export file.

In the table, Column name represents the fields that are exported in the .csv file, and Description describes the data of each field and the name of the column as displayed in the Compute Optimizer console. The EC2 instances and Auto Scaling groups columns specify if the data applies to each of those resource types. The recommendation data columns in the .csv file are numbered when multiple recommendations are generated for each resource. Ranked recommendation columns, in which <rank> is replaced with a ranking, correspond to each other. For example, recommendationOptions_1_memory, recommendationOptions_1_network, and recommendationOptions_1_vcpus correspond to each other, and are for the same recommendation.

Column name Description EC2 instances Auto Scaling groups

accountId

The AWS account ID in which the source resource was created. This column is listed as Account ID in the Compute Optimizer console.

Yes

Yes

autoScalingGroupArn

The Amazon Resource Name (ARN) of the source Auto Scaling group. This column is listed as Auto Scaling group ARN in the Compute Optimizer console.

No

Yes

autoScalingGroupName

The name of the Auto Scaling group. This column is listed as Auto Scaling group name in the Compute Optimizer console.

No

Yes

currentConfiguration_desiredCapacity

The desired capacity of the source Auto Scaling group. This column is listed as Current desired capacity in the Compute Optimizer console.

No

Yes

currentConfiguration_instanceType

The instance type of the source Auto Scaling group. This column is listed as Current instance type in the Compute Optimizer console.

No

Yes

currentConfiguration_maxSize

The maximum size of the source Auto Scaling group. This column is listed as Current maximum size in the Compute Optimizer console.

No

Yes

currentConfiguration_minSize

The minimum size of the source Auto Scaling group. This column is listed as Current minimum size in the Compute Optimizer console.

No

Yes

currentInstanceType

The instance type of the source instance. This column is listed as Current instance type in the Compute Optimizer console.

Yes

No

current_memory

The memory of the source instance, including for instances in an Auto Scaling group. This column is listed as Current memory in the Compute Optimizer console.

Yes

Yes

current_network

The network performance, or rate of data transfer, of the source instance, including for instances in an Auto Scaling group. This column is listed as Current network in the Compute Optimizer console.

Yes

Yes

current_onDemandPrice

The On-Demand price of the source instance, including for instances in an Auto Scaling group. The price listed might not reflect the actual price you pay for the instance. This column is listed as Current On-Demand price in the Compute Optimizer console.

Yes

Yes

current_standardOneYearNoUpfrontReservedPrice

The Reserved Instances, standard 1-year no upfront price of the source instance, including for instances in an Auto Scaling group. The price listed might not reflect the actual price you pay for the instance. This column is listed as Current 1-year RI price in the Compute Optimizer console.

Yes

Yes

current_standardThreeYearNoUpfrontReservedPrice

The Reserved Instances, standard 3-year no upfront price of the source instance, including for instances in an Auto Scaling group. The price listed might not reflect the actual price you pay for the instance. This column is listed as Current 3-year RI price in the Compute Optimizer console.

Yes

Yes

current_storage

The local storage volume of the source instance, including for instances in an Auto Scaling group. This column is listed as Current storage in the Compute Optimizer console.

Yes

Yes

current_vcpus

The number of vCPUs of the source instance, including for instances in an Auto Scaling group. This column is listed as Current vCPUs in the Compute Optimizer console.

Yes

Yes

finding

The finding classification for the source instance, or for the source Auto Scaling group. Instances can be classified as under-provisioned, optimized, or over-provisioned. Auto Scaling groups can be classified as not optimized, or optimized. This column is listed as Finding in the Compute Optimizer console.

Yes

Yes

instanceArn

The Amazon Resource Name (ARN) of the source instance. This column is listed as Instance ARN in the Compute Optimizer console.

Yes

No

instanceName

The name of the source instance. This column is listed as Instance name in the Compute Optimizer console.

Yes

No

lastRefreshTimestamp

The timestamp of when the resource recommendation was last refreshed. This column is listed as Last refresh timestamp in the Compute Optimizer console.

Yes

Yes

lookBackPeriodInDays

The number of preceding days for which Compute Optimizer analyzed metric data from the source resource to generate the recommendation. This column is listed as Lookback period in days in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_configuration_desiredCapacity

The desired capacity of the Auto Scaling group recommendation. This column is listed as Recommendation options desired capacity in the Compute Optimizer console.

No

Yes

recommendationOptions_<rank>_configuration_instanceType

The instance type of the Auto Scaling group recommendation. This column is listed as Recommendation options instance type in the Compute Optimizer console.

No

Yes

recommendationOptions_<rank>_configuration_maxSize

The maximum size of the Auto Scaling group recommendation. This column is listed as Recommendation options maximum size in the Compute Optimizer console.

No

Yes

recommendationOptions_<rank>_configuration_minSize

The minimum size of the Auto Scaling group recommendation. This column is listed as Recommendation options minimum size in the Compute Optimizer console.

No

Yes

recommendationOptions_<rank>_instanceType

The instance type of the instance recommendation. This column is listed as Recommendation options Instance type in the Compute Optimizer console.

Yes

No

recommendationOptions_<rank>_memory

The memory of the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommendation options memory in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_network

The network performance, or rate of data transfer, of the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommendation options network in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_onDemandPrice

The On-Demand price of the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommendation options On-Demand price in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_performanceRisk

The performance risk of the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommendation options performance risk in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_projectedUtilizationMetrics_CPU_MAXIMUM

The projected maximum CPU utilization metric of the instance recommendation, including for instances in an Auto Scaling group. This value defines the maximum CPU utilization if you had used the recommended resource during the look-back period. This column is listed as Recommendation options projected utilization metrics CPU maximum in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_projectedUtilizationMetrics_MEMORY_MAXIMUM

The projected maximum memory utilization metric of the instance recommendation, including for instances in an Auto Scaling group. This value defines the maximum memory utilization if you had used the recommended resource during the look-back period. This column is listed as Recommendation options projected utilization metrics memory maximum in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_standardOneYearNoUpfrontReservedPrice

The Reserved Instances, standard 1-year no upfront price for the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommended options 1-year RI price in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_standardThreeYearNoUpfrontReservedPrice

The Reserved Instances, standard 3-year no upfront price for the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommended options 3-year RI price in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_storage

The local storage volume of the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommendation options storage in the Compute Optimizer console.

Yes

Yes

recommendationOptions_<rank>_vcpus

The vCPUs of the instance recommendation, including for instances in an Auto Scaling group. This column is listed as Recommendation options vCPUs in the Compute Optimizer console.

Yes

Yes

recommendationsSources_<rank>_recommendationSourceArn

The Amazon Resource Name (ARN) of the source resource. This column is listed as Recommendation source ARN in the Compute Optimizer console.

Yes

No

recommendationsSources_<rank>_recommendationSourceType

The source resource type. This column is listed as Recommendation source type in the Compute Optimizer console.

Yes

No

utilizationMetrics_CPU_MAXIMUM

The maximum CPU utilization metric of the source instance, including for instances in an Auto Scaling group. This column is listed as Utilization metrics CPU maximum in the Compute Optimizer console.

Yes

Yes

utilizationMetrics_MEMORY_MAXIMUM

The maximum memory utilization metric of the source instance, including for instances in an Auto Scaling group. This column is listed as Utilization metrics memory maximum in the Compute Optimizer console.

Yes

Yes

recommendations_count

The number of recommendation included. This column is not listed in the Compute Optimizer console.

Yes

Yes

errorCode

The error code for when a recommendation was not generated for a resource. This column is not listed in the Compute Optimizer console.

Yes

Yes

errorMessage

The error message that corresponds to the error code. This column is not listed in the Compute Optimizer console.

Yes

Yes

Metadata file

A metadata .json file is output with every export job. It includes the schema information for the associated recommendations file, such as the dialect of the data, column definitions, column descriptions, and more. The file is meant to help parse the export file, and describe its contents. The metadata file is saved in the same S3 bucket and prefix that you specified for the export file.

The metadata file includes the following properties for each exported column, or field:

  • Name - The export field recommendations column name.

  • Titles - The user-friendly recommendations column name.

  • Datatype - The type of data for the column.

  • Null - The string to expect if the column is null.

  • Required - Indicates if the column data is required.

The following is an example of the information included in the metadata file.

{ "@context": [ "http://www.w3.org/ns/csvw" ], "url": "us-east-1-2020-05-18T001229Z-f264881a-bfb3-4676-9b14-8d1243599ebb.csv", "dc:title": "EC2 Instance Recommendations", "dialect": { "encoding": "utf-8", "lineTerminators": [ "\n" ], "doubleQuote": true, "skipRows": 0, "header": true, "headerRowCount": 1, "delimiter": ",", "skipColumns": 0, "skipBlankRows": false, "trim": false }, "dc:modified": { "@value": "2020-05-20", "@type": "xsd:date" }, "tableSchema": { "columns": [ { "name": "accountId", "titles": "Account ID", "datatype": "string", "null": "", "required": false }, { "name": "instanceArn", "titles": "Instance Arn", "datatype": "string", "null": "", "required": false }, { "name": "utilizationMetrics_CPU_MAXIMUM", "titles": "Cpu Maximum Utilization Metrics", "datatype": "double", "null": "", "required": false }, { "name": "recommendations_count", "titles": "Number of recommendations", "datatype": "integer", "required": true }, { "name": "recommendationOptions_1_instanceType", "titles": "Recommendation 1 Instance Type", "datatype": "integer", "null": "", "required": false }, { "name": "lastRefreshTimestamp_UTC", "titles": "Last Resfreshed Timestamp UTC", "datatype": "datetime", "format": "yyyy-MM-dd HH:mm:ss", "null": "", "required": false }, { "name": "errorCode", "titles": "Error Code", "datatype": "string", "required": true }, { "name": "errorMessage", "titles": "Error Message", "datatype": "string", "required": true } ] } }