Rich Monitoring and Logging

A cornerstone of detecting and protecting one’s environment and data is the ability to granularly monitor configurations across an enterprise and robust logging of activities occurring within an IT infrastructure. Visibility and traceability within IT environments is often hard to achieve for large on-premises operations that focus on physical separation based security controls. This design can result in fragmentation of operational views due to the lack of integration across services. This situation makes threat detection and root cause analysis challenging. AWS builds core security services that are highly integrated throughout AWS services, including monitoring and logging. AWS CloudTrail, Amazon CloudWatch, VPC Flow Logs, and AWS Config integrate across AWS services offerings, providing clear records of activities and configuration changes. The information provided by these services paints a multi-dimensional view of the operational state of the systems and data from functional, performance, and security perspectives. This comprehensive visibility can also be achieved at a lower cost compared to on-premises enterprise systems.

AWS CloudTrail provides the option to log AWS API requests for customers, regardless of whether the requests were made through the AWS Management Console, AWS SDKs, command line tools, or via other AWS services on the customer’s behalf. Each log event identifies the caller identity and called AWS API, the source IP address of the call, when the call occurred, and other parameters specific to the API. The logs can be ingested into a customer’s local security information and event management (SIEM) system for analysis or sent to other AWS analytics services like CloudWatch Logs Insights. AWS CloudTrail logs are digitally signed to prevent tampering before they are stored in Amazon S3 for customers to access. Logs can also be retained using S3 Object Lock to create strong policies that makes all users unable to delete the object. Logs are encrypted in storage, optionally under keys the customer controls in AWS KMS.

Amazon CloudWatch is used to monitor AWS resources and applications in near real-time. It can collect, track, and alarm based on metrics that are accessible via customizable dashboards or APIs. CloudWatch data are encrypted in transit and at rest. In addition, Amazon EventBridge delivers a near real-time stream of system events that describe changes to AWS resources to customers, which can set alarms and be notified of potentially unauthorized access. Rules can be implemented to match events and routed to one or more target functions or streams for further monitoring or even execution of corrective actions. For example, rules can examine incoming events, parse the incoming values, and properly route the event to any number of targets, such as email or mobile devices, ticketing queues, and issue management systems.

Configuration management is at the heart of controlling changes to an environment. Configurations that drift from their intended state present a risk to a system’s security posture. Managing and enforcing configuration states across an on-premises environment is usually difficult because the tooling to measure a system’s present state often lacks enough points of integration to offer a holistic view of the enterprise. In AWS, customers can address configuration management in multiple ways. One of the best options is to move toward an infrastructure as code (IaC) model for your environment. IaC allows you to provision, de-provision, and maintain infrastructure configuration state in consistent, repeatable, and automated manner using code. This includes being able to use secure code management practices and test automation directly on infrastructure components. One way to accomplish this with AWS is using AWS CloudFormation.

AWS CloudFormation templates can create, configure, and manage resources through use JSON or YAML. You manage these resources declared in the templates in units called AWS CloudFormation Stacks. Stacks can be composed as StackSets to manage resources across regions and accounts from single templates or sets of templates. From a monitoring standpoint, CloudFormation is integrated with CloudTrail for recording actions performed by the service. Additionally, CloudFormation can detect configuration drift between the current resource configuration from a StackSets against the expected configuration declared in the StackSets. This level of configuration management can detect unmanaged changes and allow the user to reapply the template to return the resources to the declared state.

Often deeper and broader configuration management capabilities are needed by customers to handle the many ways AWS resources can be provisioned, changed, and managed. AWS Config fills this need by providing a detailed, continuous view of the configuration of AWS resources in a customer’s AWS accounts. This includes how the resources are related to one another and how they were configured in the past so that a customer can see how the configurations and relationships change over time. AWS Config provides an AWS resource inventory, configuration history, and configuration change notifications across regions and accounts. These capabilities, along with advanced querying and customizable rules, enables security and governance insights and workflow automation for AWS resources. 

Another linchpin for deep monitoring and logging is traffic flow visibility. VPC Flow Logs are a feature whereby a customer can capture information about the IP traffic going to and from network interfaces in their VPC. Flow log data can be published as records to Amazon CloudWatch Logs and Amazon S3 for further analysis. A flow log can be created for an entire VPC, a subnet, or a single network interface. In addition to Flow Logs, VPC also allows full packet capture when useful or necessary using its Traffic Mirroring feature. These two features work well together, VPC Flow Logs for routine network logging, and temporarily enabling Traffic Mirroring when circumstances require it.

Dealing with the volumes of logging data can be cumbersome for some customers so many choose to ease monitoring and analysis of logs by using Amazon GuardDuty, the AWS-managed threat detection offering. GuardDuty is a service that provides threat detection and continuous network security monitoring by consuming and analyzing many of the data sources mentioned here such as Flow Logs and CloudTrail logs, plus internal AWS DNS logs and threat intelligence feeds. GuardDuty applies machine learning, behavioral anomaly analysis, and other detection techniques to identify threats across network activity.