Cloud-based computing usually involves groups of AWS resources, such as EC2 instances, EBS volumes, and so on, which must be created and managed collectively. For example, a web application typically requires application servers, database servers, load balancers, and so on. This group of instances is typically called a stack; a simple application server stack might look something like the following.
In addition to creating the instances and installing the necessary packages, you typically need a way to distribute applications to the application servers, monitor the stack's performance, manage security and permissions, and so on.
AWS OpsWorks provides a simple and flexible way to create and manage stacks and applications. It supports a standard set of components—including application servers, database servers, load balancers, and more—that you can use to assemble your stack. These components all come with a standard configuration and are ready to run. However, you aren't limited to the standard components and configurations. AWS OpsWorks gives you the tools to customize the standard package configurations, install additional packages, and even create your own custom components. AWS OpsWorks also provides a way to manage related AWS resources, such as Elastic IP addresses and Amazon EBS volumes.
Here's what a basic PHP application server stack might look with AWS OpsWorks. It consists of a group of PHP application servers running behind an Elastic Load Balancing load balancer, with a backend MySQL database server.
Although relatively simple, this stack shows all the key AWS OpsWorks features. Here's how it's put together.
The stack is the core AWS OpsWorks component. It is basically a container for AWS resources—Amazon EC2 instances, Amazon EBS volumes, Elastic IP addresses, and so on—that have a common purpose and would be logically managed together. The stack helps you manage these resources as a group and also defines some default configuration settings, such as the instances' operating system and AWS region. If you want to isolate some stack components from direct user interaction, you can run the stack in a VPC.
You define the stack's components by adding one or more layers. A layer is basically a blueprint that specifies how to configure a set of Amazon EC2 instances for a particular purpose, such as serving applications or hosting a database server. You assign each instance to at least one layer, which determines what packages are to be installed on the instance, how they are configured, whether the instance has an Elastic IP address or Amazon EBS volume, and so on.
AWS OpsWorks includes a set of built-in layers that support the following standard scenarios:
Application server: Java App Server, Node.js App Server, PHP App Server, Rails App Server, Static Web Server
Database server: Amazon RDS and MySQL
Load balancer: Elastic Load Balancing, HAProxy
Monitoring server: Ganglia
In-memory key-value store: Memcached
If the built-in layers don't quite meet your requirements, you can customize or extend them by modifying packages' default configurations, adding custom Chef recipes to perform tasks such as installing additional packages, and more. You can also customize layers to work with AWS services that are not natively supported, such as using Amazon RDS as a database server. If that's still not enough, you can create a fully custom layer, which gives you complete control over which packages are installed, how they are configured, how applications are deployed, and more.
AWS OpsWorks depends on Chef recipes to handle tasks such as installing packages on instances, deploying apps, configuring load balancers, and so on. One of the key AWS OpsWorks features is a set of lifecycle events—Setup, Configure, Deploy, Undeploy, and Shutdown—which automatically run the appropriate recipes at the appropriate time on each instance.
Each layer has a set of recipes for each lifecycle event that run on the layer's instances to handle a variety of tasks for that event and layer. For example, after an instance that belongs to a PHP App Server layer finishes booting, AWS OpsWorks does the following.
Runs the layer's Setup recipes, which perform tasks such as installing and configuring a PHP application server.
Runs the layer's Deploy recipes, which deploy the layer's applications from the repository to the instance and handle related tasks such as restarting the service.
Runs the Configure recipes on every instance in the stack so each instance can adjust its configuration as needed to accommodate the new instance.
For example, on an instance running a load balancer, a Configure recipe could modify the load balancer's configuration to include the new instance.
If an instance belongs to multiple layers, AWS OpsWorks runs the recipes for each layer so you can, for example, have an instance that supports a PHP application server and a MySQL database server.
If you have implemented your own custom recipes, you don't have to run them manually. Just assign each recipe to the appropriate layer and event, and AWS OpsWorks will run them for you as needed.
Each layer has at least one instance, which represents an Amazon EC2 instance and defines its basic configuration, such as operating system and size. Other configuration settings, such as Elastic IP addresses or Amazon EBS volumes, are defined by the instance's layer. The layer's recipes complete the configuration by performing tasks such as installing and configuring packages and deploying apps.
When you start an instance, AWS OpsWorks launches an Amazon EC2 instance using the configuration settings specified by the instance and its layer. After the Amazon EC2 has booted, AWS OpsWorks installs an agent that handles communication between the instance and the service and runs the appropriate recipes in response to lifecycle events.
AWS OpsWorks supports the following instance types, which are characterized by how they are started and stopped.
24/7 instances are started manually and run until you stop them.
Load-based instances are automatically started and stopped by AWS OpsWorks, based on specified load metrics, such as CPU utilization. They allow your stack to automatically adjust the number of instances to accommodate variations in incoming traffic.
Time-based instances are run by AWS OpsWorks on a specified daily and weekly schedule. They allow your stack to automatically adjust the number of instances to accommodate predictable usage patterns.
AWS OpsWorks also supports autoheal instances. If an agent stops communicating with the service, AWS OpsWorks automatically stops and restarts the instance.
You store applications and related files in a repository such as an Amazon S3 bucket. Each application is represented by an app, which specifies the application type and contains the information that AWS OpsWorks needs to deploy the application from the repository to your instances. You can deploy apps in two ways:
Automatically—When you start an app server instance, AWS OpsWorks automatically deploys all apps of the appropriate type; Java apps are deployed to the Java App Server layer's instances, and so on.
Manually—If you have a new app or want to update an existing one, you can manually deploy it to your online instances.
When you deploy an app, AWS OpsWorks runs the Deploy recipes on the stack's instances. The app server layer's Deploy recipes download the app from the repository to the instance and perform related tasks such as configuring the server and restarting the daemon. You typically have AWS OpsWorks run the Deploy recipes on the entire stack, which allows the other layers' instances to modify their configuration appropriately. However, you can limit deployment to a subset of instances if, for example, you want to test a new app before deploying it to every app server instance.
The built-in layers provide standard functionality that is sufficient for many purposes. However, you might customize them a bit. AWS OpsWorks provides a variety of ways to customize layers to meet your specific requirements:
You can modify how AWS OpsWorks configures packages by overriding attributes that represent the various configuration settings, or by even overriding the templates used to create configuration files.
You can extend an existing layer by providing your own custom recipes to perform tasks such as running scripts or installing and configuring nonstandard packages.
You can create a fully custom layer by providing a set of recipes that handle all the tasks of installing packages, deploying apps, and so on.
You package your custom recipes and related files in one or more cookbooks and store the cookbooks in a repository such Amazon S3. You can run your custom recipes manually, but AWS OpsWorks also lets you automate the process by supporting a set of five lifecycle events:
Setup occurs on a new instance after it successfully boots.
Configure occurs on all of the stack's instances when an instance enters or leaves the online state.
Deploy occurs when you deploy an app.
Undeploy occurs when you delete an app.
Shutdown occurs when you stop an instance.
Each layer comes with a set of built-in recipes assigned to each of these events. When a lifecycle event occurs on a layer's instance, AWS OpsWorks runs the associated recipes. For example, when a Deploy event occurs on an app server instance, AWS OpsWorks runs the layer's built-in Deploy recipes to download the app.
You can run custom recipes automatically by assigning them to a layer's lifecycle events. AWS OpsWorks then automatically runs them for you after the built-in recipes have finished. For example, if you want to install an additional package on a layer's instances, write a recipe to install the package and assign it to the layer's Setup event. AWS OpsWorks will run it automatically on each of the layer's new instances, after the built-in Setup recipes are finished.
With AWS OpsWorks, you can use any of your account's Elastic IP address and Amazon EBS volume resources in a stack. You can use the AWS OpsWorks console or API to register resources with a stack, attach registered resources to or detach them from instances, and move resources from one instance to another.
AWS OpsWorks integrates with AWS Identity and Access Management (IAM) to provide robust ways of controlling how users access AWS OpsWorks, including the following:
How individual users can interact with each stack, such as whether they can create stack resources such as layers and instances, or whether they can use SSH to connect to a stack's EC2 instances.
How AWS OpsWorks can act on your behalf to interact with AWS resources such as Amazon EC2 instances.
How apps that run on AWS OpsWorks instances can access AWS resources such as Amazon S3 buckets.
How to manage public SSH keys and connect to an instance with SSH.
AWS OpsWorks provides several features to help you monitor your stack and troubleshoot issues with your stack and any custom recipes:
CloudWatch monitoring, which is summarized for your convenience on the OpsWorks Monitoring page.
CloudTrail logs, which record API calls made by or on behalf of AWS OpsWorks in your AWS account.
A Ganglia master layer, which you can use to collect and display detailed monitoring data for the instances in your stack.
An event log, which lists all events in your stack.
Chef logs that describe the details of what transpired for each lifecycle event on each instance, such as which recipes were run and what errors occurred.
In addition to the graphical console, AWS OpsWorks also supports a command-line interface (CLI) and SDKs for multiple languages that can be used to perform any operation. Consider these features:
The AWS OpsWorks CLI is part of the AWS CLI, and can be used to perform any operation from the command-line.
The AWS CLI supports multiple AWS services and can be installed on Windows, Linux, or OS X systems.
AWS OpsWorks is included in AWS Tools for Windows PowerShell and can be used to perform any operation from a PowerShell command line.
You can also use AWS CloudFormation templates to provision stacks. For some examples, see AWS OpsWorks Snippets.
The following video walks you through a basic example of how to use AWS OpsWorks.
For a more detailed introduction to AWS OpsWorks that covers a broader range of features than the video, see Create a Simple PHP Application Server Stack.
For a brief overview of the AWS OpsWorks documentation, see Documentation Roadmap.