Assessment reports - AWS Audit Manager

Assessment reports

An AWS Audit Manager assessment report summarizes the selected evidence that was collected for an assessment. It also contains links to evidence PDF files that contain the supporting evidence. The specific contents, organization, and naming conventions of assessment reports depend on the parameters that you choose when generating the report.

When you generate an assessment report from an active AWS Audit Manager assessment, you can select which evidence you want to include in the report. The assessment report is then placed in your specified Amazon S3 bucket. For more information, see Generating an assessment report.

Assessment reports are designed to help you select and compile the evidence that is relevant for your audit, but they do not assess the compliance of the evidence itself. Instead, AWS Audit Manager simply provides all of the selected evidence as output.

How to navigate an assessment report

Assessment reports begin with a high-level overview. This includes a summary of the assessment report itself, along with a summary of the assessment that the report was created from.

After you've read the overview, you can use the table of contents (TOC) to navigate the rest of the report. Choose any hyperlinked control set or control in the TOC to jump directly to that item and read more details. From here, you can either return to the TOC to choose a different control or control set, or continue reading to see the detailed breakdown of a control's evidence.

When you're ready to review evidence for a control, you can do so by choosing the hyperlinked evidence name. For automated evidence, choosing the hyperlinked evidence name opens a new PDF file with a summary and further details about that evidence. These evidence PDF files are included as part of the assessment report package that you download from AWS Audit Manager. For manual evidence, the hyperlink takes you to the S3 bucket that contains the manual evidence.

Tip

The breadcrumb navigation at the top of each page shows your current location in the assessment report as you browse controls sets and controls. Select the hyperlinked TOC to navigate back to the TOC at any time.

Report sections

The following sections provide information about each section of the assessment report. Choose a topic to learn more about that section and reference its data definitions.

Note

When you see a hyphen (-) next to any of the data attributes in the following sections, this indicates that the value of that attribute is null, or a value does not exist.

Cover page

The cover page includes the name of the assessment report. It also displays the date and time that the report was generated, along with the account ID of the user who generated the assessment report.

The cover page is formatted as follows. AWS Audit Manager replaces the placeholders with the information relevant to your report.

Assessment report name

Report generated on MM/DD/YYYY at HH:MM:SS AM/PM UCT by AccountID from the following assessment: Assessment name

Overview

The overview contains two parts: a summary of the report itself, and a summary of the assessment that the report was generated from.

Assessment report summary

The following information is included in the assessment report summary.

  • Assessment report name – The name of the report.

  • Assessment report description – The description that's entered by the audit owner when they generate the report.

  • Date generated – The date when the report was generated. The time is represented in Coordinated Universal Time (UTC).

  • Control sets – The number of control sets in the report.

  • Controls – The total number of controls in the report.

  • AWS Region in scope – The AWS Region where the report was created.

  • AWS accounts in scope – The list of AWS account IDs that are included in the scope of the report.

  • AWS services in scope – The list of AWS services that are included in the scope of the report.

  • Compliance check status – The total number of compliance check issues that are found in the report.

  • Assessment report selection – The number of evidence items that are selected for inclusion in the report.

Assessment summary

The following information is included in the assessment summary.

  • Assessment name – The name of the assessment that the report was generated from.

  • Audit owner – The AWS Identity and Access Management (IAM) user or role for the audit owner.

  • Date created – The date when the assessment was created. The time is represented in UTC.

  • Last updated – The date when the assessment was last updated. The time is represented in UTC.

  • Assessment status – The status of the assessment at the time when the assessment report was generated.

  • AWS Region in scope – The AWS Region that's in the scope of the assessment.

  • AWS accounts in scope – The list of AWS account IDs that are in the scope of the assessment.

  • AWS services in scope – The list of AWS services that are in the scope of the assessment.

  • Framework name – The name of the framework that the assessment was created from.

  • Framework description – The optional description of the framework that the assessment was created from.

  • Framework type – Specifies whether the framework is a standard or custom framework.

  • Compliance type – The name of the compliance standard or regulation that the framework supports.

Table of contents

The table of contents displays the full contents of the assessment report. The contents are grouped and organized based on the control sets that are included in the assessment. Controls are always nested underneath their respective control set.

Choose any item in the table of contents to navigate directly to that section of the report. You can either choose a control set or go directly to a control.

Control set page

You can use the control set summary to familiarize yourself with the control set before choosing a particular control to review.

Control set summary

The following information is included in the control set summary.

  • Control set name – The name of the control set.

  • Control set status – The review status of the control set at the time when the report was generated.

  • Total controls – The total number of controls in the control set.

  • Compliance check status – The number of compliance check issues found for this control set, out of all of the evidence that was selected for inclusion in the assessment report for the given control set.

  • Assessment report selection – The number of evidence items from this control set that were included in the report.

  • Controls – The list of controls that are part of the control set. Choose the hyperlinked control to go directly to the page in the report that contains more information about that control.

Control page

The control page contains two parts: a summary of the control itself and a summary of the related evidence for that control that was included in the report.

Control summary

The following information is included in the control summary.

  • Control name – The name of the control.

  • Control set – The name of the control set that the control belongs to.

  • Control description – The description of the control.

  • Testing information – The recommended testing procedures for this control.

  • Action plan – The recommended actions to perform if the control is not fulfilled.

  • First evidence collection date – The date and time when the first piece of evidence was collected for this control. The time is represented in UTC.

  • Last evidence collection date – The date and time when the last piece of evidence was collected for this control. The time is represented in UTC.

  • Compliance check status – The number of compliance check issues that were found for this control's evidence.

  • Assessment report selection – The number of evidence items related to this control that were included in the assessment report.

Assessment report selection

In the assessment report selection table, a list of evidence folders is displayed with the following data columns.

  • Evidence name – Displays the evidence grouped by folders. These folders are then organized and named by the date on which the evidence was collected. Following each folder name in bold is a list of hyperlinked evidence names.

    • Automated evidence names begin with date of the automated evidence collection, followed by a unique identifier and the _auto suffix (for example, [HH-MM-SSAM/PMUTC]_[abcdefghij]_[auto]). For automated evidence, the hyperlinked name opens a new PDF file with a summary and further details about that evidence.

    • Manual evidence names begin with the date of the manual upload, followed by a unique identifier, the first 10 characters of the filename, and the file extension (for example, [HH-MM-SSAM/PMUTC]_[abcdefghij]_[ManualEvid.csv]). For manual evidence, the hyperlinked name takes you to the S3 bucket that contains the manual evidence object.

  • Compliance check status – Next to each evidence folder name is the total number of compliance check issues for that folder. For each evidence row under that folder, the Compliance check status column displays the result of the corresponding compliance check.

    • For automated evidence that's collected from AWS Security Hub, a Passed, Failed, Warning, or Not applicable result is reported directly from Security Hub. For more information about these statuses, see Determining the overall status of a control from its findings in the Security Hub User Guide.

    • For automated evidence that's collected from AWS Config, a Compliant, Non compliant, or Not applicable result is reported directly from AWS Config. For more information about these statuses, see Compliance in the AWS Config API Reference.

    • For automated evidence that's collected from AWS CloudTrail and API calls, and for all manual evidence, Not applicable appears.

Evidence summary page

The following information is included in the evidence summary.

  • Evidence name – The name of the evidence. The name is based on the date when the evidence was created or uploaded on.

  • Evidence folder – The name of the folder where the evidence is located. The name is based on the date when the evidence was created on, recorded in the [YYYY-MM-DD] format.

  • Evidence description – The description of the evidence. It includes the related AWS account and the source where the evidence was collected from.

  • Assessment report name – The name of the report.

  • Assessment name – The name of the assessment that the report was generated from.

  • Framework name – The name of the framework that the assessment was created from.

  • Framework description – The optional description for the framework.

  • Framework type – Specifies whether the framework is a standard or custom framework.

  • Compliance type – The compliance standard or regulation that the framework supports.

  • Control name – The name of the control that the evidence supports.

  • Control set name – The name of the control set that the related control belongs to.

  • Control description – The description of the control that the evidence supports.

  • Testing information – The recommended testing procedures for the control.

  • Action plan – The recommended actions to perform if the control is not fulfilled.

  • AWS Region – The name of the Region that's associated with the evidence.

  • IAM ID – The ARN of the IAM user or role that's associated with the evidence.

  • AWS account – The AWS account ID that's associated with the evidence.

  • AWS service – The name of the AWS service that's associated with the evidence.

  • Resources included – The AWS resources that were assessed to generate the evidence. This attribute is not applicable for compliance check evidence from AWS Config. For this evidence type, you can find all of the resources tabulated in the Evidence detail page of the evidence PDF.

  • Event name – The name of the evidence event.

  • Event time – The time when the evidence event occurred.

  • Data source – The name of the AWS service that the evidence was collected from.

  • Evidence by type – The category of the evidence.

    • Compliance check evidence is collected from AWS Config or AWS Security Hub.

    • User activity evidence is collected from AWS CloudTrail logs.

    • Configuration data evidence is collected from snapshots of other AWS services.

    • Manual evidence is evidence that you upload manually.

  • Compliance check status – The evaluation status for evidence that falls under the compliance check category.

    • For automated evidence that's collected from AWS Security Hub, a Passed, Failed, Warning, or Not applicable result is reported directly from Security Hub. For more information about these statuses, see Determining the overall status of a control from its findings in the Security Hub User Guide.

    • For automated evidence that's collected from AWS Config, a Compliant, Non compliant, or Not applicable result is reported directly from AWS Config. For more information about these statuses, see Compliance in the AWS Config API Reference.

    • For automated evidence that's collected from AWS CloudTrail and API calls, and for all manual evidence, Not applicable appears.

Evidence detail page

The evidence detail page shows the name of the evidence and an evidence detail table. This table provides a detailed breakdown of each element of the evidence so that you can understand the data and validate that it's correct.

Depending on the data source of the control, the contents of the evidence detail page vary. For example, evidence details from an API call consist of a list of the API parameters in a tabular format, and the corresponding responses for each element.

Tip

The breadcrumb navigation at the top of each page shows your current location as you browse evidence details. Select the hyperlinked evidence summary name to navigate back to the evidence summary at any time.

Assessment report integrity check

When you generate assessment reports for your audit, AWS Audit Manager produces a report file checksum so that you can validate that the report remains unaltered. You can download the report to share evidence with your auditors.

To validate the integrity of a report, use the ValidateAssessmentReportIntegrity API provided by AWS Audit Manager.

Troubleshooting assessment reports

Use the information here to help you troubleshoot and fix issues that you might encounter when working with assessment reports in Audit Manager.

My assessment report failed to generate

Your assessment report might have failed to generate for a number of reasons. You can start to troubleshoot this issue by checking the most frequent causes. Use the following checklist to get started.

  1. Check if any of your AWS Region information doesn't match up:

    1. Does the AWS Region of your S3 bucket match the AWS Region of your assessment? The S3 bucket that you use as your assessment report destination must be in the same AWS Region as your assessment. For instructions on how to change the S3 bucket, see AWS Audit Manager settings, Assessment report destination.

    2. Does the AWS Region of your customer managed key match the AWS Region of your assessment? If you provided a customer managed key for data encryption, it must be in the same AWS Region as your assessment. For instructions on how to change the KMS key, see AWS Audit Manager settings, Data encryption.

  2. Check the permissions of the S3 bucket that you’re using as the assessment report destination:

    1. Does the IAM entity that’s generating the assessment report have the necessary permissions for the S3 bucket? The IAM entity must have the required S3 bucket permissions to publish reports in that bucket. We provide an example policy that you can use. For instructions on how to specify a different S3 bucket, see AWS Audit Manager settings, Assessment report destination.

    2. Does the S3 bucket have a bucket policy that requires server-side encryption (SSE) using SSE-KMS? If yes, the KMS key that's used in that bucket policy must match the KMS key that's specified in your Audit Manager data encryption settings. If you didn't configure a KMS key in your Audit Manager settings, and your S3 bucket policy requires SSE, ensure that the bucket policy allows SSE-S3. For instructions on how to configure the assessment report destination and the KMS key that's used for data encryption, see AWS Audit Manager settings.

If you’re still unable to successfully generate an assessment report, review the following issues on this page.

I followed the checklist above, and my assessment report still failed to generate

Audit Manager can support up to approximately 22,000 evidence items in a single assessment report. If you try to generate a report that contains more evidence than this, the operation might fail.

If you encounter this issue, we recommend that you generate multiple assessment reports as a workaround. This will allow you to export evidence from your assessment into more manageable-sized batches.

I’m unable to unzip the assessment report

If you can't unzip the assessment report on Windows, it's likely that Windows Explorer can't extract it because its file path has several nested folders or long names. This is because, under the Windows file naming system, the folder path, file name, and file extension can’t exceed 259 characters. Otherwise, this results in a Destination Path Too Long error.

To resolve this issue, try moving the zip file to the parent folder of its current location. You can then try again to unzip it from there. Alternatively, you can also try shortening the name of the zip file or extracting it to a different location that has a shorter file path.

I get an access denied error when I try to generate a report

You will get an access denied error if your assessment was created by a delegated administrator account that the KMS key that's specified in your Audit Manager settings doesn't belong to. To avoid this error, when you designate a delegated administrator for Audit Manager, make sure that the delegated administrator account has access on the KMS key that you provided when setting up Audit Manager.

You might also receive an access denied error if you don't have write permissions for the S3 bucket that you're using as your assessment report destination.

If you get an access denied error, make sure that you meet the following requirements:

  • Your AWS KMS key in your Audit Manager settings gives permissions to the delegated administrator. You can configure this by following the instructions in Allowing users in other accounts to use a KMS key in the AWS Key Management Service Developer Guide. For instructions on how to review and change your encryption settings in Audit Manager, see Data encryption.

  • You have a permissions policy that grants you write access for the S3 bucket that you're using as the assessment report destination. More specifically, your permissions policy contains an s3:PutObject action, specifies the ARN of the S3 bucket, and includes the key used to encrypt your assessment reports. For an example policy that you can use, see Identity-based policy examples for AWS Audit Manager.

Note

If you change your Audit Manager data encryption settings, these changes apply to the new assessments that you create moving forward. This includes any assessment reports that you create from your new assessments.

The changes don't apply to existing assessments that you created before you changed your encryption settings. This includes new assessment reports that you create from existing assessments, in addition to existing assessment reports. Existing assessments—and all their assessment reports—continue to use the old KMS key. If the IAM identity that’s generating the assessment report doesn’t have permissions to use the old KMS key, you can grant permissions at the key policy level.

My assessment report generation is stuck in In progress status, and I'm not sure how this impacts my billing

Assessment report generation has no impact on billing. You're only billed based on the evidence that your assessments collect. For more information about pricing, see AWS Audit Manager Pricing.