How Amazon GuardDuty uses its data sources - Amazon GuardDuty

How Amazon GuardDuty uses its data sources

To detect unauthorized and unexpected activity in your AWS environment, GuardDuty analyzes and processes data from the sources described in this topic. GuardDuty uses these data sources to detect anomalies involving the following AWS resource types: IAM access keys, EC2 instances, S3 buckets, and Amazon EKS resources. While in transit from these data sources to GuardDuty, all of the log data is encrypted. GuardDuty extracts various fields from these logs for profiling and anomaly detection, and then discards the logs.

The following sections describe how GuardDuty uses each supported data source.

Important

Some of the data sources might not be available in other AWS Regions. For more information, see Regions and endpoints.

AWS CloudTrail event logs

AWS CloudTrail provides you with a history of AWS API calls for your account, including API calls made using the AWS Management Console, the AWS SDKs, the command line tools, and certain AWS services. CloudTrail also allows you to identify which users and accounts called AWS APIs for services that support CloudTrail, the source IP address that the calls were made from, and when the calls occurred. For more information, see the AWS CloudTrail User Guide. GuardDuty can monitor both CloudTrail management events, and optionally, CloudTrail data events for S3.

When you enable GuardDuty, it immediately starts analyzing your CloudTrail event logs. It consumes CloudTrail management and S3 data events directly from CloudTrail through an independent and duplicated stream of events. There is no additional charge for GuardDuty to access CloudTrail events.

GuardDuty does not manage your CloudTrail events or affect your existing CloudTrail configurations in any way. To manage access and retention of your CloudTrail events directly, you must use the CloudTrail service console or API. For more information, see Viewing Events with CloudTrail Event History.

How GuardDuty handles AWS CloudTrail global events

Another important detail about the way GuardDuty uses CloudTrail as a data source is the handling and processing of CloudTrail global events. For most services, events are recorded in the Region where the action occurred. For global services such as IAM, AWS Security Token Service, Amazon S3, Amazon CloudFront, and RouteĀ 53, events are delivered to any trail that includes global services, and are logged as occurring in the US East (N. Virginia) Region. For more information, see About Global Service Events.

GuardDuty processes all events that come into a Region, including global events that CloudTrail sends to all Regions. This allows GuardDuty to maintain user and role profiles in each Region, and enables it to accurately detect credentials that are being maliciously used across Regions.

We highly recommend that you enable GuardDuty in all supported AWS Regions. This enables GuardDuty to generate findings about unauthorized or unusual activity even in Regions that you are not actively using. This also enables GuardDuty to monitor AWS CloudTrail events for global AWS services. If GuardDuty is not enabled in all supported Regions, its ability to detect activity that involves global services is reduced.

AWS CloudTrail management events

Management events are also known as control plane events. These events provide insight into management operations that are performed on resources in your AWS account.

The following are examples of CloudTrail management events that GuardDuty monitors:
  • Configuring security (IAM AttachRolePolicyAPI operations)

  • Configuring rules for routing data (Amazon EC2 CreateSubnet API operations)

  • Setting up logging (AWS CloudTrail CreateTrail API operations)

AWS CloudTrail data events for S3

Data events, also known as data plane operations, provide insight into the resource operations performed on or within a resource. They are often high-volume activities.

The following are examples of CloudTrail data events for S3 that GuardDuty can monitor:
  • GetObject API operations

  • PutObject API operations

  • ListObjects API operations

  • DeleteObject API operations

S3 data event monitoring is enabled by default for new accounts. However, this data source is optional and can be enabled or disabled for any account or Region at any time. For more information about configuring Amazon S3 as a data source, see Amazon S3 Protection in Amazon GuardDuty.

Kubernetes audit logs

Kubernetes audit logs capture sequential actions within your Amazon EKS cluster, including activities from users, applications using the Kubernetes API, and the control plane. Audit logging is a component of all Kubernetes clusters. For more information, see Auditing in the Kubernetes documentation.

Amazon EKS allows Kubernetes audit logs to be ingested as Amazon CloudWatch Logs through the EKS control plane logging feature. When you enable EKS Protection in GuardDuty, GuardDuty immediately starts analyzing Kubernetes audit logs for your Amazon EKS resources. It consumes audit log events directly from the Amazon EKS control plane logging feature through an independent and duplicative stream of flow logs. This process does not require any additional set up or affect any existing Amazon EKS control plane logging configurations that you might have.

GuardDuty doesn't manage your Amazon EKS control plane logging or make Kubernetes audit logs accessible in your account if you have not enabled them for Amazon EKS. To manage access to and retention of your Kubernetes audit logs, you must configure the Amazon EKS control plane logging feature. For more information, see Enabling and disabling control plane logs in the Amazon EKS documentation.

Kubernetes audit log monitoring is enabled by default all new GuardDuty accounts. However, this data source is optional and can be enabled or disabled for any account or region at any time. For more information about configuring Kubernetes audit logs a data source, see EKS Protection in Amazon GuardDuty.

VPC Flow Logs

The VPC Flow Logs feature of Amazon VPC captures information about the IP traffic going to and from network interfaces within your environment.

When you enable GuardDuty, it immediately starts analyzing your VPC flow logs data. It consumes VPC flow log events directly from the VPC Flow Logs feature through an independent and duplicative stream of flow logs. This process does not affect any existing flow log configurations that you might have.

GuardDuty doesn't manage your flow logs or make them accessible in your account. To manage access to and retention of your flow logs, you must configure the VPC Flow Logs feature.

With no additional charge, GuardDuty can access the flow logs. GuardDuty only charges for the amount of flow logs data processed (in GB) to generate a finding. GuardDuty optimizes costs by applying smart filters and analyzing subset of flow logs relevant to threat detection. However, enabling flow logs for retention or use in your account falls under the existing pricing.

DNS logs

If you use AWS DNS resolvers for your Amazon EC2 instances (the default setting), then GuardDuty can access and process your request and response DNS logs through the internal AWS DNS resolvers. If you use another DNS resolver, such as OpenDNS or GoogleDNS, or if you set up your own DNS resolvers, then GuardDuty cannot access and process data from this data source.

When you enable GuardDuty, it immediately starts analyzing your DNS logs from an independent stream of data. This data stream is separate from the data provided through the RouteĀ 53 Resolver query logging feature. Configuration of this feature does not effect GuardDuty analysis.

Elastic Block Storage (EBS) volume

GuardDuty Malware Protection scans and detects malware on Amazon EBS volumes attached to your Amazon EC2 instances.

When GuardDuty generates a finding indicative of malware in an EC2 instance or container workload, and you have enabled Malware Protection, GuardDuty will create snapshots of the EBS volumes attached to the potentially impacted EC2 instance or container workload in your account. Using the permissions provisioned to GuardDuty through the Service-linked role permissions for GuardDuty Malware Protection, GuardDuty will create encrypted replica EBS volumes from that snapshot in the GuardDuty's service account. GuardDuty will then scan the replica EBS volumes for malware.

Note

GuardDuty Malware Protection doesn't scan EBS volumes that are encrypted using Amazon EBS encryption. For more information, see Understanding how Malware Protection works in GuardDuty.

After the scan completes, GuardDuty deletes the encrypted replica EBS volumes and the snapshots of your EBS volumes. If malware is found and you've turned on the Snapshots retention setting, the snapshots of your EBS volumes won't get deleted and are automatically retained in your AWS account. When no malware is found, the snapshots of your EBS volumes will not be retained, irrespective of the snapshots retention setting.

GuardDuty will retain each replica EBS volume that it scans for up to one day, unless and to the extent that there is a service outage or failure with a replica EBS volume and its malware scan, at which point, GuardDuty will retain such an EBS volume for no more than seven days. The extended retention period is to triage and address the outage or failure. GuardDuty Malware Protection will delete the replica EBS volume after the outage or failure is addressed or once the extended retention period lapses.