Remediating security issues discovered by GuardDuty

Amazon GuardDuty generates findings that indicate potential security issues. In this release of GuardDuty, the potential security issues indicate either a compromised EC2 instance or container workload, or a set of compromised credentials in your AWS environment. The following sections describe the recommended remediation steps for these scenarios. If there are alternative remediation scenarios they will be described in the entry for that specific finding type. You can access the full information about a finding type by selecting it from the Active findings types table.

Topics

• Remediating a compromised Amazon EC2 instance
• Remediating a compromised S3 bucket
• Remediating a compromised ECS cluster
• Remediating compromised AWS credentials
• Remediating a compromised standalone container
• Remediating EKS Audit Log Monitoring findings discovered by GuardDuty
• Remediating EKS Runtime Monitoring findings
• Remediating a compromised database
• Remediating a compromised Lambda function

Remediating a compromised Amazon EC2 instance

Follow these recommended steps to remediate a compromised EC2 instance in your AWS environment:

1. Isolate the impacted Amazon EC2 instance.

   Investigate the potentially compromised instance for malware and remove any discovered malware. You may use On-demand malware scan to identify malware in the potentially compromised EC2 instance, or check AWS Marketplace to see if there are helpful partner products to identify and remove malware.

2. Identify the source of the suspicious activity

   If malware is detected, then based on the finding type in your account, identify and stop the potentially unauthorized activity on your EC2 instance. This may require actions such as closing any open ports, changing access policies, and upgrading applications to correct vulnerabilities.

   If you are unable to identify and stop unauthorized activity on your EC2 instance, we recommend that you terminate the compromised EC2 instance and replace it with a new instance as needed. The following are additional resources for securing your EC2 instances:

   • Security and Networking sections in Best practices for Amazon EC2

   • Amazon EC2 security groups for Linux instances and Amazon EC2 security groups for Windows instances

   • Security in Amazon EC2

   • Tips for securing your EC2 instances (Linux).

   • AWS security best practices

   • Infrastructure Domain Incidents on AWS

3. Browse AWS re:Post

   Browse AWS re:Post at https://forums.aws.amazon.com/index.jspa for further assistance.

4. Submit a technical support request

   If you are a premium support package subscriber, you can submit a technical support request.

Remediating a compromised S3 bucket

Follow these recommended steps to remediate a compromised Amazon S3 bucket in your AWS environment:

1. Identify the affected S3 resource.

   A GuardDuty finding for S3 will list an S3 bucket, the bucket's Amazon Resource Name (ARN) and a bucket owner in the finding details.

2. Identify the source of the suspicious activity and the API call used.

   The API call used will be listed as API in the finding details. The source will be an IAM principal (either an IAM role, user, or account) and identifying details will be listed in the finding. Depending on the source type, Remote IP address or source domain info will be available and can help you evaluate whether the source was authorized. If the finding involved credentials from an EC2 instance the details for that resource will also be included.

3. Determine whether the call source was authorized to access the identified resource.

   For example consider the following:

   • If an IAM user was involved, is it possible their credentials have been compromised? See the following section on Remediating Compromised AWS Credentials.

   • If an API was invoked from a principal that has no prior history of invoking this type of API, does this source need access permissions for this operation? Can the bucket permissions be further restricted?

   • If the access was seen from the user name ANONYMOUS_PRINCIPAL with user type of AWSAccount this indicates the bucket is public and was accessed. Should this bucket be public? If not, review the security recommendations below for alternative solutions to sharing S3 resources.

   • If the access was though a successful PreflightRequest call seen from the user name ANONYMOUS_PRINCIPAL with user type of AWSAccount this indicates the bucket has a cross-origin resource sharing (CORS) policy set. Should this bucket have a CORS policy? If not, ensure the bucket is not inadvertently public and review the security recommendations below for alternative solutions to sharing S3 resources. For more information on CORS see Using cross-origin resource sharing (CORS) in the S3 user guide.

4. Determine whether the S3 bucket contains sensitive data.

   Use Amazon Macie to determine whether the S3 bucket contains sensitive data, such as personally identifiable information (PII), financial data, or credentials. If automated sensitive data discovery is enabled for your Macie account, review the S3 bucket's details to gain a better understanding of your S3 bucket's contents. If this feature is disabled for your Macie account, we recommend turning it on to expedite your assessment. Alternatively, you can create and run a sensitive data discovery job to inspect the S3 bucket's objects for sensitive data. For more information, see Discovering sensitive data with Macie.

If the access was authorized, you can ignore the finding. The https://console.aws.amazon.com/guardduty/ console allows you to set up rules to entirely suppress individual findings so that they no longer appear. For more information, see Suppression rules.

If you determine that your S3 data has been exposed or accessed by an unauthorized party review the following S3 security recommendations to tighten permissions and restrict access. Appropriate remediation solutions will depend on the needs of your specific environment.

These are some recommendations based on specific S3 access needs:

• For a centralized way to limit public access to your S3 data use S3 block public access. Block public access settings can be enabled for access points, buckets, and AWS Accounts through four different settings to control granularity of access. For more information see S3 Block Public Access Settings.

• AWS Access policies can be used to control how IAM users can access your resources or how your buckets can be accessed. For more information see Using Bucket Policies and User Policies.

  Additionally you can use Virtual Private Cloud (VPC) endpoints with S3 bucket policies to restrict access to specific VPC endpoints. For more information see Example Bucket Policies for VPC Endpoints for Amazon S3

• To temporarily allow access to your S3 objects to trusted entities outside your account you can create a Presigned URL through S3. This access is created using your account credentials and depending on the credentials used can last 6 hours to 7 days. For more information see Generating presigned URLs with S3.

• For use cases that require that sharing of S3 objects between different sources you can use S3 Access Points to create permission sets that restrict access to only those within your private network. For more information see Managing data access with Amazon S3 access points.

• To securely grant access to your S3 resources to other AWS Accounts you can use an access control list (ACL), for more information see Managing S3 Access with ACLs.

For a full overview of S3 security options see S3 Security best practices.

Remediating a compromised ECS cluster

Follow these recommended steps to remediate a compromised ECS cluster in your AWS environment:

1. Identify the affected ECS cluster.

   The GuardDuty Malware Protection finding for ECS provides the ECS cluster details in the finding's details panel.

2. Evaluate the source of malware

   Evaluate if the detected malware was in the container's image. If malware was in the image, identify all other tasks which are running using this image. For information on running tasks, see ListTasks.

3. Isolate the impacted tasks

   Isolate the impacted tasks by denying all ingress and egress traffic to the task. A deny all traffic rule may help stop an attack that is already underway, by severing all connections to the task.

If the access was authorized, you can ignore the finding. The https://console.aws.amazon.com/guardduty/ console allows you to set up rules to entirely suppress individual findings so that they no longer appear. For more information, see Suppression rules.

Remediating compromised AWS credentials

Follow these recommended steps to remediate compromised credentials in your AWS environment:

1. Identify the affected IAM entity and the API call used.

   The API call used will be listed as API in the finding details. The IAM entity (either an IAM role or user) and it's identifying information will be listed in the Resource section of a finding's details. The type of IAM entity involved can be determined by the User Type field, the name of the IAM entity will be in the User name field. The type of IAM entity involved in the finding can also be determined by the Access key ID used.

   For keys beginning with AKIA:

   This type of key is a long term customer-managed credential associated with an IAM user or AWS account root user. For information about managing access keys for IAM users, see Managing access keys for IAM users.

   For keys beginning with ASIA:

   This type of key is a short term temporary credential generated by AWS Security Token Service. These keys exists for only a short time and cannot be viewed or managed in the AWS Management Console. IAM roles will always use AWS STS credentials, but they can also be generated for IAM Users, for more information on AWS STS see IAM: Temporary security credentials.

   If a role was used the User name field will indicate the name of the role used. You can determine how the key was requested with AWS CloudTrail by examining the sessionIssuer element of the CloudTrail log entry, for more information see IAM and AWS STS information in CloudTrail.

2. Review permissions for the IAM entity.

   Open the IAM console, depending on the type of entity used, choose the Users or Roles tab, and locate the affected entity by typing the identified name into the search field. Use the Permission and Access Advisor tabs to review effective permissions for that entity.

3. Determine whether the IAM entity credentials were used legitimately.

   Contact the user of the credentials to determine if the activity was intentional.

   For example, find out if the user did the following:

   • Invoked the API operation that was listed in the GuardDuty finding

   • Invoked the API operation at the time that is listed in the GuardDuty finding

   • Invoked the API operation from the IP address that is listed in the GuardDuty finding

If this activity is a legitimate use of the AWS credentials, you can ignore the GuardDuty finding. The https://console.aws.amazon.com/guardduty/ console allows you to set up rules to entirely suppress individual findings so that they no longer appear. For more information, see Suppression rules.

If you can't confirm if this activity is a legitimate use, it could be the result of a compromise to the particular access key, the IAM user's sign-in credentials, or possibly the entire AWS account. If you suspect your credentials have been compromised, review the information in the My AWS account may be compromised article to remediate this issue.

Remediating a compromised standalone container

1. Isolate the container

   To identify the malicious container workload, follow the steps below:

   • Open the GuardDuty console at https://console.aws.amazon.com/guardduty/.

   • On the Findings page, choose the corresponding finding to open the findings panel.

   • In the findings panel, under the Resource affected section, you can view the container's ID and Name.

   Isolate this container from other container workloads.

2. Pause the container

   Suspend all the processes in your container. For information on how to freeze your container, see Pause a container.

   Stop the container

   If the step above fails, and the container doesn't pause, stop the container from running. If you've enabled the Snapshots retention feature, GuardDuty will retain the snapshots of your EBS volumes that contain malware.

   For information on how to stop the container, see Stop a container.

3. Evaluate the presence of malware

   Evaluate if malware was in the container's image.

If the access was authorized, you can ignore the finding. The https://console.aws.amazon.com/guardduty/ console allows you to set up rules to entirely suppress individual findings so that they no longer appear. The GuardDuty console allows you to set up rules to entirely suppress individual findings so that they no longer appear. For more information, see Suppression rules.