Troubleshooting evidence collection issues - AWS Audit Manager

Troubleshooting evidence collection issues

You can use the information on this page to resolve common evidence collection issues in Audit Manager.

I created an assessment but I can’t see any evidence yet

If you can't see any evidence, it's likely that you either didn't wait at least 24 hours after you created the assessment or that there's a configuration error.

We recommend that you check the following:

  1. Make sure that more than 24 hours passed since you created the assessment. Automated evidence becomes available 24 hours after you create the assessment.

  2. Make sure that you’re using Audit Manager in the same AWS Region as the AWS service that you’re expecting to see evidence for.

  3. If you expect to see compliance check evidence from AWS Config and AWS Security Hub, make sure that both the AWS Config and Security Hub consoles are displaying results for these checks. The AWS Config and Security Hub results should be displaying in the same AWS Region that you use Audit Manager in.

If you still can't see any evidence in your assessment and it's not because of one of these issues, consider checking for the other issues that are described on this page.

My assessment isn’t collecting any compliance check evidence from AWS Security Hub

This issue can be caused if you missed some configuration steps in your AWS Security Hub settings.

If you're using a single AWS account, you must enable AWS Config and the PCI DSS security standard for your account.

If you're using Organizations, you must do the following:

  • Enable AWS Config and the PCI DSS security standard for every member account.

  • Designate the same administrator account in Security Hub and in Audit Manager.

Make sure that you configured your Security Hub settings as follows.

Before you enable any security standards in Security Hub, make sure that you enabled AWS Config and configured resource recording. For more information, see Enabling and configuring AWS Config in the AWS Security Hub User Guide. Then, follow this procedure to configure your Security Hub settings for Audit Manager.

To configure Security Hub settings for a single account

  1. Sign in to the AWS Management Console and open the AWS Security Hub console at https://console.aws.amazon.com/securityhub/.

  2. In the left navigation pane, choose Security standards.

  3. Under PCI DSS v3.2.1, choose Enable to enable the PCI DSS security standard for your account. By default, the AWS CIS Foundations Benchmark standard and the AWS Foundational Best Practices standard are already enabled. For more information, see Enabling a security standard in the AWS Security Hub User Guide.

Before you enable any security standards in Security Hub, make sure that you enabled AWS Config and configured resource recording for your organization. For more information, see Enabling and configuring AWS Config in the AWS Security Hub User Guide. Then, follow this procedure to configure your Security Hub settings for Audit Manager.

To configure Security Hub settings for an organization

  1. Sign in to the AWS Management Console and open the AWS Security Hub console at https://console.aws.amazon.com/securityhub/.

  2. Using your AWS Organizations management account, designate an account as the delegated administrator for Security Hub. Make sure that the delegated administrator account that you designate in Security Hub is the same one that you designated in Audit Manager. For more information, see Designating a Security Hub administrator account in the AWS Security Hub User Guide.

  3. Using your Organizations delegated administrator account, go to Settings, Accounts and enable your organization accounts as Security Hub member accounts. For more instructions, see Enabling member accounts from your organization in the AWS Security Hub User Guide.

  4. Enable the PCI DSS security standard for every member account of the organization. By default, the AWS CIS Foundations Benchmark standard and the AWS Foundational Best Practices standard are already enabled. For more information, see Enabling a security standard in the AWS Security Hub User Guide.

My assessment isn’t collecting evidence from another AWS service

If an AWS service isn't selected as in scope for your assessment, Audit Manager doesn't collect evidence from resources related to that service. This is also the case if an AWS service is selected but you haven't enabled it in your environment.

If you created your assessment from a custom framework, you can edit the services in scope for your assessment. You can then specify additional AWS services that you want to collect evidence from. After you add these services, evidence becomes available after 24 hours.

Note

If you created your assessment from a standard framework, the list of AWS services in scope is preselected and can’t be edited. This is because when you create an assessment from a standard framework, Audit Manager automatically maps and selects the relevant data sources and services for you. The selection is made based on the requirements of the standard framework. Note that, for standard frameworks that contain manual controls only, no AWS services are in scope.

The workaround for editing the AWS services in scope while still creating an assessment based on a standard framework is to customize the standard framework. By using this workaround, you can use the framework that you customized to create a new assessment. In this assessment, you can then specify which AWS services are in scope.

My evidence is generated at different intervals, and I don’t understand how often it’s being collected

The controls in Audit Manager assessments are mapped to a combination of data sources. Each data source has a different evidence collection frequency. As a result, there’s no one-size-fits-all answer for how often evidence is collected. Some data sources evaluate compliance, whereas others only capture the resource state and change data without a compliance determination.

The following is a summary of the different data sources and their evidence collection frequency.

Data source Description Evidence collection frequency When this control is active

AWS CloudTrail

Tracks a specified user activity that's needed in your audit.

Continual

Audit Manager assesses your CloudTrail logs and filters the relevant logs based on your keyword. The processed logs are converted into User activity evidence.

AWS Security Hub

Captures a snapshot of your resource security posture by reporting the result of a compliance check from Security Hub.

Based on the schedule of the Security Hub check (typically around every 12 hours)

Audit Manager assesses the Security Hub findings that are associated with this Security Hub check. The processed data is converted into Compliance check evidence.

AWS Config

Captures a snapshot of your resource security posture by reporting the result of a compliance check from AWS Config.

Based on the settings defined in the AWS Config rule Audit Manager assesses the AWS Config findings that are associated with this AWS Config rule. The processed data is converted into Compliance check evidence.
API calls

Takes a snapshot of your resource configuration directly through an API call to the specified AWS service.

Daily, weekly, or monthly Audit Manager makes the API call based on the frequency that you specify, and assesses the results. The results are converted into Configuration data evidence.

Regardless of the evidence collection frequency, new evidence is collected automatically for as long as the assessment is active. For more information, see Evidence collection frequency.

To learn more about control data sources, see Supported control data sources for automated evidence and Changing the evidence collection frequency for a control.