Troubleshooting evidence finder issues

Use the information on this page to resolve common evidence finder issues in Audit Manager.

I can't enable evidence finder

Common reasons why you can't enable evidence finder include the following situations:

You're missing permissions

If you’re trying to enable evidence finder for the first time, make sure that you have the required permissions. These permissions allow you to create and manage an event data store in CloudTrail Lake, which is necessary to support evidence finder search queries. The permissions also allow you to run search queries in evidence finder.

If you need help with permissions, contact your AWS administrator. If you’re an AWS administrator, you can copy the required permission statement and attach it to an IAM policy.

You're using your Organizations management account

Keep in mind that you can't use your management account to enable evidence finder. Sign in as the delegated administrator account, and try again.

You previously disabled evidence finder

Re-enabling evidence finder isn't currently supported. If you previously disabled evidence finder, you can't enable it again.

I enabled evidence finder, but I don't see past evidence in my search results

When you enable evidence finder, it takes up to 7 days for all of your past evidence data to become available.

During this 7-day period, an event data store is backfilled with your past two years’ worth of evidence data. This means that if you use evidence finder immediately after you enable it, not all results are available until the backfill is complete.

For instructions on how to check the status of the data backfill, see Confirming the status of evidence finder.

I can't disable evidence finder

This could be caused by one of the following reasons.

You're missing permissions

If you’re trying to disable evidence finder, make sure that you have the required permissions. These permissions allow you to update and delete an event data store in CloudTrail Lake, which is necessary to disable evidence finder.

If you need help with permissions, contact your AWS administrator. If you’re an AWS administrator, you can copy the required permission statement and attach it to an IAM policy.

A request to enable evidence finder is still in progress

When you request to enable evidence finder, we create an event data store to support evidence finder queries. You can't disable evidence finder while the event data store is being created.

To proceed, wait until the event data store is created, and try again. For more information, see Confirming the status of evidence finder.

You already requested to disable evidence finder

When you request to disable evidence finder, we delete the event data store that's used for evidence finder queries. If you try again to disable evidence finder while the event data store is being deleted, you get an error message.

In this case, no action is needed. Wait for the event data store to be deleted. As soon as this is complete, evidence finder is disabled. For more information, see Confirming the status of evidence finder.

My search query fails

A failed search query could be caused by one of the following reasons.

You're missing permissions

Verify that the user has the required permissions to run search queries and access the search results. Specifically, you need permissions for the following CloudTrail actions:

• StartQuery

• DescribeQuery

• CancelQuery

• GetQueryResults

If you need help with permissions, contact your AWS administrator. If you’re an AWS administrator, you can copy the required permission statement and attach it to an IAM policy.

You're running the maximum number of queries

You can run up to 5 queries at a time. If you're running the maximum number of concurrent queries, this results in a MaxConcurrentQueriesException error. If you get this error message, wait a minute for some queries to finish, and then run the query again.

Your query statement has a validation error

If you're using the API or CLI to perform the CloudTrail Lake StartQuery operation, make sure that your queryStatement is valid. If the query statement has validation errors, incorrect syntax, or unsupported keywords, this results in an InvalidQueryStatementException.

For more information about writing a query, see Create or edit a query in the AWS CloudTrail User Guide.

For examples of valid syntax, review the following query statement examples that can be used to query an Audit Manager event data store.

Example 1: Investigate evidence and its compliance status

This example finds evidence with any compliance status across all assessments in account, within a specified date range.

SELECT eventData.evidenceId, eventData.resourceArn, eventData.resourceComplianceCheck FROM $EDS_ID WHERE eventTime > '2022-11-02 00:00:00.000' AND eventTime < '2022-11-03 00:00:00.000'

Example 2: Determine non-compliant evidence for a control

This example finds all non-compliant evidence in a specified date range for a specific assessment and control.

SELECT * FROM $EDS_ID WHERE eventData.assessmentId = '11aa33bb-55cc-77dd-99ee-ff22gg44hh66' AND eventTime > '2022-10-27 22:05:00.000' AND eventTime < '2022-11-03 22:05:00.000' AND eventData.resourceComplianceCheck IN ('NON_COMPLIANT','FAILED','WARNING') AND eventData.controlId IN ('aa11bb22-cc33-dd44-ee55-ff66gg77hh88')

Example 3: Count evidence by name

This example lists the total evidence for an assessment in a specified date range, grouped by name and ordered by evidence count.

SELECT eventData.eventName as eventName, COUNT(*) as totalEvidence FROM  $EDS_ID WHERE eventData.assessmentId = '11aa33bb-55cc-77dd-99ee-ff22gg44hh66' AND eventTime > '2022-10-27 22:05:00.000' AND eventTime < '2022-11-03 22:05:00.000' GROUP BY eventData.eventName ORDER BY totalEvidence DESC

Example 4: Explore evidence by data source and service

This example finds all evidence in a specified date range for a specific data source and service.

SELECT * FROM $EDS_ID WHERE eventTime > '2022-10-27 22:05:00.000' AND eventTime < '2022-11-03 22:05:00.000' AND eventData.service IN ('dynamodb') AND eventData.dataSource IN ('AWS API calls')

Example 5: Explore compliant evidence by data source and control domain

This example finds compliant evidence for specific control domains, where the evidence comes from a data source that isn't AWS Config.

 SELECT * FROM $EDS_ID WHERE eventData.resourceComplianceCheck IN ('PASSED','COMPLIANT') AND eventData.controlDomainName IN ('Logging and monitoring','Data security and privacy') AND eventData.dataSource NOT IN ('AWS Config')

Other API exceptions

The StartQuery API might fail for several other reasons. For a complete list of possible errors and descriptions, see StartQuery Errors in the AWS CloudTrail API Reference.

I can’t generate multiple assessment reports from my search results

This error is caused by running too many concurrent CloudTrail Lake queries.

This error can occur if you group your search results and attempt to immediately generate assessment reports for each line item in your grouped results. Make sure to give enough time between requests. Each time you get results or generate an assessment report, each action invokes a query. You can only run up to 5 queries at one time.

To prevent this error, make sure that you aren’t generating too many assessment reports at one time. If you’re running the maximum number of concurrent queries, a MaxConcurrentQueriesException error is returned. If you get this error message, wait a few minutes for your in-progress assessment reports to complete.

You can check the status from the assessment reports page in the Audit Manager console. After your reports are complete, return to your grouped results in evidence finder. You can then continue to get the results and generate an assessment report for each line item.

I can't add individual search results to an assessment report

Adding individual search results to an assessment report isn’t currently supported. All search results are included in the report.

If you only want to include specific search results in the report, we recommend that you edit your current search filters. This way, you can narrow down your results to target only the evidence that you want to add to a report.

Not all of my evidence finder results are included in the assessment report

When you generate an assessment report, there are limits for how much evidence you can add. The limit is based on the AWS Region of your assessment, the Region of the S3 bucket that's used as your assessment report destination, and whether your assessment uses a customer managed AWS KMS key.

1. The limit is 22,000 for same-Region reports (where the S3 bucket and assessment are in the same AWS Region)

2. The limit is 3,500 for cross-Region reports (where the S3 bucket and assessment are in different AWS Regions)

3. The limit is 3,500 if the assessment uses a customer managed KMS key

If you exceed this limit, the report is still created. However, Audit Manager adds only the first 3,500 or 22,000 evidence items to the report.

To prevent this issue, we recommend that you edit your current search filters. This way, you can reduce your search results by targeting a smaller amount of evidence. If needed, you can repeat this method and generate multiple assessment reports instead of one larger report.

I want to generate an assessment report from my search results, but my query statement is failing

If you're using the CreateAssessmentReport API and your query statement returns a validation exception, check the table below for guidance on how to fix it.

Note

Even if a query statement works in CloudTrail, the same query might not be valid for assessment report generation in Audit Manager. This is because of some differences in query validation between the two services.

Clause	Issue	Solution	Notes
SELECT	The SELECT clause contains a column name	Remove the SELECT clause and replace with SELECT eventJson.	Only SELECT eventJson is supported. This validation is handled by Audit Manager.
FROM	The FROM clause contains an invalid event data store ID or The provided event data store ID doesn’t match the event data store ID in your Audit Manager settings	Remove the FROM clause and replace with FROM edsID, where the value of edsID matches the event data store ID that's specified in your Audit Manager settings. You can retrieve the ARN of the event data store from your Audit Manager settings. For more information, see GetSettings in the AWS Audit Manager API Reference.	This validation is handled by Audit Manager.
GROUP BY	A GROUP BY clause is present in the query	Remove the GROUP BY clause.	This validation is handled by Audit Manager.
HAVING	A HAVING clause is present in the query	Remove the HAVING clause.	This validation is handled by Audit Manager.
LIMIT	The LIMIT clause contains a value that exceeds the maximum allowed limit	If the LIMIT clause exists, ensure that its value is equal to or less than the maximum supported limit: For same-Region reports, the limit is 22,000 For cross-Region reports, the limit is 3,500 For reports where the related assessment uses a customer managed AWS KMS key, the limit is 3,500	In the console, there’s no limit to the number of evidence results that can be returned. However, when generating an assessment report, a limit applies to the amount of evidence that you can include. If no LIMIT value is provided in your query statement, the default maximum limits are applied. This validation is handled by Audit Manager.
ORDER BY	The ORDER BY clause contains Aggregate functions or Aliases that aren’t present in the SELECT clause	Ensure that the ORDER BY clause doesn’t contain any conditions using Aggregate functions or Aliases.	This validation is handled by the CloudTrail StartQuery API.
WHERE	The WHERE clause contains more than one assessmentId or The WHERE clause contains an assessmentId that doesn’t match the assessmentId in your createAssessmentReport request or The WHERE clause contains an unsupported column name	Ensure that only one assessmentID is specified, and that it matches the assessmentId parameter that you specified in the createAssessmentReport API request. Remove any unsupported column names.	This validation is handled by the CloudTrail StartQuery API.

Examples

The following examples show how you can use the queryStatement parameter when calling the CreateAssessmentReport operation. Before you use these queries, replace the placeholder text with your own edsId and assessmentId values.

Example 1: Create a report (same-Region limit applies)

This example creates a report that includes results for S3 buckets created between January 22-23rd, 2022.

SELECT eventJson FROM 12345678-abcd-1234-abcd-123456789012 WHERE eventData.assessmentId = '11aa33bb-55cc-77dd-99ee-ff22gg44hh66' AND eventTime > '2022-01-22 00:00:00.000' AND eventTime < '2022-01-23 00:00:00.000' AND eventName='CreateBucket' LIMIT 22000

Example 2: Create a report (cross-Region limit applies)

This example creates a report that includes all results for the specified event data store and assessment, with no date range specified.

SELECT eventJson FROM 12345678-abcd-1234-abcd-123456789012 WHERE eventData.assessmentId = '11aa33bb-55cc-77dd-99ee-ff22gg44hh66' LIMIT 7000

Example 3: Create a report (under the default limit)

This example creates a report that includes all results for the specified event data store and assessment, with a limit that’s under the default maximum.

SELECT eventJson FROM 12345678-abcd-1234-abcd-123456789012 WHERE eventData.assessmentId = '11aa33bb-55cc-77dd-99ee-ff22gg44hh66' LIMIT 2000