AWS OpsWorks
User Guide (API Version 2013-02-18)
Did this page help you?  Yes | No |  Tell us about it...
Next »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

What is AWS OpsWorks?

Cloud-based computing usually involves groups of AWS resources, such as Amazon EC2 instances and Amazon Relational Database Service (RDS) instances, which must be created and managed collectively. For example, a web application typically requires application servers, database servers, load balancers, and so on. This group of instances is typically called a stack; a simple application server stack might look something like the following.

In addition to creating the instances and installing the necessary packages, you typically need a way to distribute applications to the application servers, monitor the stack's performance, manage security and permissions, and so on.

AWS OpsWorks provides a simple and flexible way to create and manage stacks and applications.

  • Linux-based stacks – AWS OpsWorks provides a standard set of components—including application servers, database servers, load balancers, and more—that you can use to assemble your stack.

    These components all come with a standard configuration and are ready to run. However, you aren't limited to the standard components and configurations. AWS OpsWorks gives you the tools to customize the standard package configurations, install additional packages, and even create your own custom components. AWS OpsWorks also provides a way to manage related AWS resources, such as Elastic IP addresses and Amazon EBS volumes.

  • Window-based stacks – AWS OpsWorks provides some standard components, such as load balancers and you can create custom components to support functionality such as Internet Information Server (IIS) instances.

Here's how that basic application server stack might look with AWS OpsWorks. It consists of a group of application servers running behind an Elastic Load Balancing load balancer, with a backend Amazon RDS database server.

Although relatively simple, this stack shows all the key AWS OpsWorks features. Here's how it's put together.

Stacks

The stack is the core AWS OpsWorks component. It is basically a container for AWS resources—Amazon EC2 instances, Amazon RDS database instances, and so on—that have a common purpose and should be logically managed together. The stack helps you manage these resources as a group and also defines some default configuration settings, such as the instances' operating system and AWS region. If you want to isolate some stack components from direct user interaction, you can run the stack in a VPC.

Layers

You define the stack's constituents by adding one or more layers. A layer represents a set of Amazon EC2 instances that serve a particular purpose, such as serving applications or hosting a database server.

For Linux stacks, AWS OpsWorks includes the following built-in layers,

  • Application server: Java App Server, Node.js App Server, PHP App Server, Rails App Server, Static Web Server, AWS Flow (Ruby)

  • Database server: MySQL

  • Load balancer: HAProxy

  • Monitoring server: Ganglia

  • In-memory key-value store: Memcached

The built-in layers are basically blueprints that specify how to configure a set of Amazon EC2 instances for a particular purpose. Each layer has at least one instance and the layer configuration determines what packages are installed, how they are configured, whether the instance has an Elastic IP address or Amazon EBS volume, and so on.

If these layers don't quite meet your requirements, you can customize or extend them by modifying packages' default configurations, adding custom Chef recipes to perform tasks such as installing additional packages, and more.

For all stacks, AWS OpsWorks includes service layers, which represent the following AWS services.

  • Amazon Relational Database Service

  • Elastic Load Balancing

  • Amazon EC2 Container Service

You can also create fully custom layers, which give you complete control over which packages are installed, how they are configured, how applications are deployed, and more. Custom layers are especially useful for Windows stacks, where you can use them to configure instances to support software such as Internet Information Server (IIS).

Recipes and LifeCycle Events

Layers depend on Chef recipes to handle tasks such as installing packages on instances, deploying apps, running scripts, and so on. One of the key AWS OpsWorks features is a set of lifecycle events—Setup, Configure, Deploy, Undeploy, and Shutdown—which automatically run a specified set of recipes at the appropriate time on each instance.

Each layer can have a set of recipes assigned to each lifecycle event, which handle a variety of tasks for that event and layer. For example, after an instance that belongs to a built-in PHP App Server layer finishes booting, AWS OpsWorks does the following.

  1. Runs the layer's Setup recipes, which perform tasks such as installing and configuring a PHP application server.

  2. Runs the layer's Deploy recipes, which deploy the layer's applications from a repository to the instance and perform related tasks, such as restarting the service.

  3. Runs the Configure recipes on every instance in the stack so each instance can adjust its configuration as needed to accommodate the new instance.

    For example, on an instance running a load balancer, a Configure recipe could modify the load balancer's configuration to include the new instance.

If an instance belongs to multiple layers, AWS OpsWorks runs the recipes for each layer so you can, for example, have an instance that supports a PHP application server and a MySQL database server.

If you have implemented custom recipes, you can assign each recipe to the appropriate layer and event and AWS OpsWorks automatically runs them for you at the appropriate time. You can also run custom recipes manually, at any time.

Instances

An instance represents a single computing resource, such as an Amazon EC2 instance. It defines the resource's basic configuration, such as operating system and size. Other configuration settings, such as Elastic IP addresses or Amazon EBS volumes, are defined by the instance's layers. The layer's recipes complete the configuration by performing tasks such as installing and configuring packages and deploying apps.

You can use AWS OpsWorks to create instances and add them to a layer. When you start the instance, AWS OpsWorks launches an Amazon EC2 instance using the configuration settings specified by the instance and its layer. After the Amazon EC2 instance has finished booting, AWS OpsWorks installs an agent that handles communication between the instance and the service and runs the appropriate recipes in response to lifecycle events.

AWS OpsWorks supports the following instance types, which are characterized by how they are started and stopped.

  • 24/7 instances are started manually and run until you stop them.

  • Time-based instances are run by AWS OpsWorks on a specified daily and weekly schedule.

    They allow your stack to automatically adjust the number of instances to accommodate predictable usage patterns.

  • Load-based instances are automatically started and stopped by AWS OpsWorks, based on specified load metrics, such as CPU utilization.

    They allow your stack to automatically adjust the number of instances to accommodate variations in incoming traffic. Load-based instances are available only for Linux-based stacks.

AWS OpsWorks supports instance autohealing. If an agent stops communicating with the service, AWS OpsWorks automatically stops and restarts the instance.

You can also can incorporate Linux-based computing resources into a stack that was created outside of AWS OpsWorks.

  • Amazon EC2 instances that you created directly by using the Amazon EC2 console, CLI, or API.

  • On-premises instances running on your own hardware, including instances running in virtual machines.

After you have registered one of these instances, it becomes an AWS OpsWorks instance and you can manage it in much the same way as instances that you create with AWS OpsWorks.

Apps

You store applications and related files in a repository, such as an Amazon S3 bucket. Each application is represented by an app, which specifies the application type and contains the information that is needed to deploy the application from the repository to your instances, such as the repository URL and password. When you deploy an app, AWS OpsWorks triggers a Deploy event, which runs the Deploy recipes on the stack's instances. For application server layers, the Deploy recipes download the application code from the repository and perform related tasks, such as restarting services. For other layer types, the Deploy recipes can perform tasks such as modifying configurations to accommodate the new application.

You can deploy apps in the following ways:

  • Automatically—When you start instances, AWS OpsWorks automatically runs the instance's Deploy recipes.

  • Manually—If you have a new app or want to update an existing one, you can manually run the online instances' Deploy recipes.

You typically have AWS OpsWorks run the Deploy recipes on the entire stack, which allows the other layers' instances to modify their configuration appropriately. However, you can limit deployment to a subset of instances if, for example, you want to test a new app before deploying it to every app server instance.

Customizing your Stack

The built-in layers support standard functionality that is sufficient for many purposes. However, you might want to customize them a bit. AWS OpsWorks provides a variety of ways to customize built-in layers to meet your specific requirements:

  • You can modify how AWS OpsWorks configures packages by overriding attributes that represent the various configuration settings, or by even overriding the templates used to create configuration files.

  • You can extend an existing layer by providing your own custom recipes to perform tasks such as running scripts or installing and configuring nonstandard packages.

All stacks can include one or more custom layers, which start with only a minimal set of recipes. You add functionality to the layer by implementing custom recipes to handle tasks such as installing packages, deploying apps, and so on. You package your custom recipes and related files in one or more cookbooks and store the cookbooks in a repository such Amazon S3 or Git.

You can run custom recipes manually, but AWS OpsWorks also lets you automate the process by supporting a set of five lifecycle events:

  • Setup occurs on a new instance after it successfully boots.

  • Configure occurs on all of the stack's instances when an instance enters or leaves the online state.

  • Deploy occurs when you deploy an app.

  • Undeploy occurs when you delete an app.

  • Shutdown occurs when you stop an instance.

Each layer can have any number of recipes assigned to each event. When a lifecycle event occurs on a layer's instance, AWS OpsWorks runs the associated recipes. For example, when a Deploy event occurs on an app server instance, AWS OpsWorks runs the layer's Deploy recipes to download the app or perform related tasks.

Each built-in layer has a set of recipes assigned to each of these events. You can assign custom recipes to a layer's lifecycle events, and AWS OpsWorks automatically runs them for you after any built-in recipes have finished. For example, if you want to install a package on a layer's instances, write a recipe to install the package and assign it to the layer's Setup event. AWS OpsWorks will run it automatically on each of the layer's newly booted instances, after any built-in Setup recipes are finished.

Resource Management

You can incorporate other AWS resources, such as Elastic IP addresses, into your stack. You can use the AWS OpsWorks console or API to register resources with a stack, attach registered resources to or detach them from instances, and move resources from one instance to another.

Security and Permissions

AWS OpsWorks integrates with AWS Identity and Access Management (IAM) to provide robust ways of controlling how users access AWS OpsWorks, including the following:

  • How individual users can interact with each stack, such as whether they can create stack resources such as layers and instances, or whether they can use SSH or RDP to connect to a stack's Amazon EC2 instances.

  • How AWS OpsWorks can act on your behalf to interact with AWS resources such as Amazon EC2 instances.

  • How apps that run on AWS OpsWorks instances can access AWS resources such as Amazon S3 buckets.

  • How to manage users' public SSH keys and RDP passwords and connect to an instance.

Monitoring and Logging

AWS OpsWorks provides several features to help you monitor your stack and troubleshoot issues with your stack and any custom recipes. For all stacks:

  • AWS OpsWorks provides a set of custom CloudWatch metrics for Linux stacks, which are summarized for your convenience on the Monitoring page.

    AWS OpsWorks supports the standard CloudWatch metrics for Windows stacks. You can monitor them with the CloudWatch console.

  • CloudTrail logs, which record API calls made by or on behalf of AWS OpsWorks in your AWS account.

  • An event log, which lists all events in your stack.

  • Chef logs that detail what transpired for each lifecycle event on each instance, such as which recipes were run and which errors occurred.

Linux-based stacks can also include a Ganglia master layer, which you can use to collect and display detailed monitoring data for the instances in your stack.

CLI, SDK, and AWS CloudFormation Templates

In addition to the console, AWS OpsWorks also supports a command-line interface (CLI) and SDKs for multiple languages that can be used to perform any operation. Consider these features:

  • The AWS OpsWorks CLI is part of the AWS CLI, and can be used to perform any operation from the command-line.

    The AWS CLI supports multiple AWS services and can be installed on Windows, Linux, or OS X systems.

  • AWS OpsWorks is included in AWS Tools for Windows PowerShell and can be used to perform any operation from a Windows PowerShell command line.

  • The AWS OpsWorks SDK is included in the AWS SDKs, which can be used by applications implemented in: Java, JavaScript (browser-based and Node.js), .NET, PHP, Python (boto), or Ruby.

You can also use AWS CloudFormation templates to provision stacks. For some examples, see AWS OpsWorks Snippets.

Where to Start

The following video walks you through a basic example of how to use AWS OpsWorks.

For a more detailed introduction to AWS OpsWorks that covers a broader range of features than the video, see Getting Started with Linux Stacks and Getting Started: Windows.

For a brief overview of the AWS OpsWorks documentation, see Documentation Roadmap.