Tutorial: Hello AWS OpsWorks!

This tutorial will show you how to use the AWS Flow Framework for Ruby layer for AWS OpsWorks to deploy and run the Hello World sample application that is described in detail in Hello World.

Amazon SWF support for AWS OpsWorks#

Amazon SWF now provides a dedicated layer in AWS OpsWorks that simplifies deployment of workflows and activities written using AWS Flow Framework for Ruby. Using AWS OpsWorks with Amazon SWF, you can easily set up a worker fleet that is cloud-deployable and can use advanced Amazon EC2 features such as load-based auto scaling.

Amazon SWF support for AWS OpsWorks includes updates to the Opsworks console, allowing you to deploy workflow and activity workers from AWS OpsWorks. It also includes updates to the AWS Flow Framework for Ruby to make it easy to specify the details necessary to spawn workers with a simple JSON file, registering any necessary workflow and activity types and starting the activity and workflow workers. This component is called the runner, and is provided by a new command-line utility: aws-flow-ruby.

The typical steps for deploying a new AWS Flow Framework for Ruby application on AWS OpsWorks are the following:

  1. Develop your AWS Flow Framework for Ruby workflows and activities normally.

  2. Test your application by using the runner to check that your workflow runs as expected. AWS Flow Framework for Ruby version 2.0.1 or greater is required to run this step.

    Note

    AWS Flow Framework for Ruby version 2.4.0 introduces changes that are incompatible with the current version of the flow layer in AWS OpsWorks. For now, you should use a version of the framework previous to 2.4.0.

  3. Set up your application on AWS OpsWorks using the AWS Management Console by creating a stack, layer, and application to deploy.

  4. Deploy your application using AWS OpsWorks and monitor your workflow's progress.

The following sections walk through the full set of steps to learn how to configure and use the runner to test how your application will run with AWS OpsWorks. However, if you're interested only in learning how to set up and deploy a working AWS Flow Framework for Ruby application with AWS OpsWorks, you can skip ahead to Deploying and Running Hello World on AWS OpsWorks.

Developing and Testing an AWS Flow Framework for Ruby Application using AWS OpsWorks#

In this section, we'll use AWS OpsWorks and a local utility (aws-flow-ruby) to deploy and test an AWS Flow Framework for Ruby application based on the Hello World sample used in Hello World.

Prerequisites#

To deploy the sample on AWS OpsWorks, you must have an AWS account with access to both Amazon SWF and AWS OpsWorks. If you haven't yet signed up for AWS, go to http://aws.amazon.com and click the Sign Up link to get started.

To run the sample locally using the aws-flow-ruby command-line utility (frequently referred to as the runner):

  • You will need to have at least version 2.0.1 of the AWS Flow Framework for Ruby gem installed. For more information about setting up the framework, see Getting Started.
  • Make sure that you have provided your AWS credentials using the AWS CLI or by setting the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. For more information, see Providing AWS Credentials.

Get the Tutorial Source Code#

The source code used in this tutorial is provided on GitHub at:

You can view the code there, or download it using the following link:

This code is a slightly modified version of the Hello World sample from the awslabs/aws-flow-ruby-samples project on GitHub.

Once you have the sample installed in a local directory on your system, open a terminal (command-line) window and change to the directory where you unzipped the sample code.

The sample's top-level directory contains the following files and directories:

Name Description
:filename:`Gemfile` The gem specification. Sets the AWS Flow Framework gem version to use (must be at least 2.0.1)
:filename:`helloworld.json` Describes the runner (aws-flow-ruby) configuration.
:filename:`flow` directory Contains files that are used by the runner to find the activities and workflows.
:filename:`lib` directory Contains the workflow worker, activity worker, and workflow starter code that is run to start the workflow. These files are the same files that are described in Hello World, but have been modified to run in the context of the runner.

The contents of these files will be described fully in later sections of the tutorial.

Add the Required Gemfile#

AWS OpsWorks requires that you add a Gemfile to your code in the root directory of your project to identify which version of the AWS Flow Framework for Ruby gem to use when deploying and running your code.

Here is the Gemfile for the Hello World sample:

source "http://www.rubygems.org"

gem 'aws-flow', '~> 2', '>= 2.0.1'

Set Up the Runner Configuration#

To discover details about how the workflows and activities will be run, aws-flow-ruby reads a JSON-formatted configuration file, as described in aws-flow-ruby.

Note

While using AWS OpsWorks, you don't need to specify a configuration file—the AWS Flow Framework for Ruby layer creates its own configuration based on values that you specify in the AWS Management Console. However, to test your workflow setup locally before deploying it to AWS OpsWorks, you will need to create a local runner configuration file.

To set up the runner configuration:

  1. If you have not already done so, open a terminal window and change to the location where you unzipped the sample code (aws-flow-ruby-opsworks-helloworld).

  2. In the sample's root directory, create or view the file called helloworld.json. It contains the following lines:

    {
      "domain":
        {
          "name": "HelloWorld"
        },
      "workflow_workers": [
        {
          "task_list": "workflow_tasklist"
        }
      ],
      "activity_workers": [
        {
          "task_list": "activity_tasklist"
        }
      ]
    }
    

    This file specifies the domains to register (if necessary) and use for the workflow and activity workers. It also specifies the number of workflow and activity workers to spawn and the number of process forks allowed when running the activities.

  3. In the flow directory within the :filename:`aws-flow-ruby-opsworks-helloworld` directory, create or view activities.rb and workflows.rb. The runner loads these files to discover where the activity and workflow code is.

    The flow/activities.rb file provides the activity code:

    require 'lib/helloworld_activity'
    

    The flow/workflows.rb file provides the workflow code:

    require 'lib/helloworld_workflow'
    

    In the sample, these files simply require files that exist in the lib directory. The runner will look in these files for classes that extend Activities and Workflows, which it will take to be the activities and workflows to run, respectively.

Note

If you want to explicitly specify which activity and workflow classes to run, add them to the runner configuration file instead. For more information, see Runner specification file.

The contents of the files are similar to the original files from the Hello World sample, but have been simplified. The runner takes care of most of the configuration for you. Here are the contents of each:

lib/helloworld_activity.rb

require 'aws/decider'

# The HelloWorldActivity class defines a set of activities for the HelloWorld sample.
class HelloWorldActivity
  extend AWS::Flow::Activities

  # Define which activities to run.
  activity :say_hello do
    {
      version: '1.0',
    }
  end

  # This activity will say hello when invoked by the workflow
  def say_hello(name)
    puts "Hello, #{name}!"
  end
end

lib/helloworld_workflow.rb

require_relative "helloworld_activity"

# HelloWorldWorkflow class defines the workflows for the HelloWorld sample
class HelloWorldWorkflow
  extend AWS::Flow::Workflows

  # Define which workflows to run.
  workflow :hello do
    {
      version: '1.0',
      default_execution_start_to_close_timeout: 120
    }
  end

  # Create an activity client using the activity_client method to schedule
  # activities
  activity_client(:client) { { from_class: "HelloWorldActivity" } }

  # This is the entry point for the workflow
  def hello(name)
    # Use the activity client 'client' to invoke the say_hello activity
    client.say_hello(name)
  end
end

Verify the Code and Configuration#

You can now execute aws-flow-ruby locally to make sure the runner can find and launch your workflow and activity. Because AWS OpsWorks also uses the AWS Flow Framework for Ruby runner to run your workflow code, this is a good way to test your code's readiness for AWS OpsWorks.

To verify the setup locally:

  1. If you have not already done so, open a terminal window and change to the aws-flow-ruby-opsworks-helloworld directory where you cloned or unzipped the sample code.

  2. Execute the runner from within the aws-flow-ruby-opsworks-helloworld directory, passing it the name of the configuration file you created. For example:

    aws-flow-ruby -f helloworld.json
    

    This will start the workflow worker and activity worker using the domain that is specified in helloworld.json. When it runs, you will see output from the runner, such as:

    waiting on workers [4573, 4574] to complete
    

    The workflow and activity workers are waiting for Amazon SWF tasks to schedule the workflows and activities for execution. Next, you'll need to start a workflow execution to begin workflow and activity scheduling.

  3. Run the helloworld_workflow_starter.rb script to start the workflow execution:

    bundle exec ruby lib/helloworld_workflow_starter.rb helloworld.json
    

    After a short time, you should see the results of the Hello World activity running:

    Hello, AWS Flow Framework!
    

Now you're ready to run the sample on AWS OpsWorks instead of the local machine.

Deploying and Running Hello World on AWS OpsWorks#

In this section, we'll set up and deploy the AWS OpsWorks-enabled version of the Hello World application on AWS OpsWorks. If you completed the previous section, Developing and Testing an AWS Flow Framework for Ruby Application using AWS OpsWorks, you will soon note that these steps use the same settings that you used when configuring the runner. If you didn't download and test the application locally, don't worry—the application we'll be using has been set up for AWS OpsWorks already.

Set up a Stack#

First, you will need to set up an AWS OpsWorks stack. A stack may consist of a number of layers, each of which can have apps that are deployed to the layers.

To set up the stack:

  1. Sign in to the AWS Management Console and open the AWS OpsWorks console at opsworks.

  2. Add a new stack by clicking the Add Stack button or by selecting Add Stack in the Select Stack menu.

  3. You can use the default values that are provided to create your AWS OpsWorks stack, or set your own. Here are suggested values for each of the fields on the screen:

    Field Name Choose...
    Name a unique name, or leave blank to allow AWS OpsWorks to choose a name for you.
    Region US East (N. Virginia)
    VPC No VPC
    Default Availability Zone us-east-1a
    Default operating system Amazon Linux
    Default root device type Instance store
    IAM role an existing role with access to Amazon SWF, such as "aws-opsworks-service-role", or choose a new role and give that role access to Amazon SWF.
    Default SSH key an SSH key that you've created, or create a new one. For information about using key pairs to log in to an AWS OpsWorks instance, see Using SSH to Log In to a Linux Instance in the AWS OpsWorks User Guide.
    Default IAM instance profile an existing instance profile with access to Amazon SWF, such as "aws-opsworks-ec2-role", or choose a new instance profile and give the profile access to Amazon SWF.
    Hostname theme any hostname theme you like.
    Stack color any stack color you like.

    Important

    Be sure that Chef version 11.10 is selected. You can click Advanced on the Add Stack screen to see what the Chef version is or to change it. If you've already created a stack, you can change the Chef version by editing the existing stack.

  4. When you're finished configuring your layer, click the Add Stack button at the bottom of the configuration screen to create your stack and continue.

Add a Layer to the Stack#

Now that the stack is set up, add a new layer to the stack.

To add the layer:

  1. Select your new AWS OpsWorks stack if it is not already selected, and then click Add a Layer.
  2. Set the Layer Type to AWS Flow. You'll find this type in the Other category in the drop-down list.
  3. Since you selected a default IAM profile for your layer, it will be automatically selected for the layer's EC2 instance profile.
  4. When you have finished setting your layer options, click Add Layer to complete adding the layer to your stack.

Add an Instance#

Next, add an Amazon EC2 instance to the layer using the Amazon EC2 instance profile that you set. The instance provides the computing platform to run your workflow code.

To add an instance to the layer:

  1. Choose the Instances view on the left sidebar if it is not already selected.

  2. Click Add an instance to add a new instance to the stack.

    You can accept the default options (copied from your layer), or modify the Hostname and Instance Size.

    Important

    Running an Amazon EC2 instance will incur AWS costs. For information about the costs of running various Amazon EC2 instance types, see the Amazon EC2 Pricing page.

  3. Click the Add Instance button to finalize your settings and add the instance to your layer. Your instance will initially be in the stopped state.

  4. If you'll be proceeding with adding the Hello World application and running it, start your instance now. It may take a few minutes for your instance to start.

Add the Hello World Application#

Next, you will add the modified Hello World application to your stack. This will instruct AWS OpsWorks to deploy your application to the Amazon EC2 instances that it starts on your behalf.

To add the Hello World application to your stack:

  1. Click the Navigation heading and choose Apps in the list. If your browser window is large enough, you'll find the Apps link positioned on the left sidebar.

  2. Click Add an app to begin adding a new application.

  3. In the Settings section, choose a name that is memorable, such as helloworld. Verify that the Type is set to AWS Flow (Ruby).

  4. Under AWS Flow (Ruby) Settings, choose an Amazon SWF domain name to run your workflows and activities under. This doesn't need to be the same domain that you used in the local test of the Hello World application. The runner will register the domain if necessary, so feel free to choose a new domain name if you would like to.

    You can accept the default workflow retention period of 10 days.

  5. Under Run Workflows and Run Activities, set the name of the workflow and activity task lists to use. If you chose to use the same domain that you used in the local test, you should pick new workflow and activity task names so that your workflows and activities don't receive tasks from the local test.

    Note

    The workflow starter that you initiate later in Start your workflow execution will need to use the same task list name as the one you set for your workflow worker in order to start the workflow.

  6. Under Data Sources, choose None for the Data Source Type.

  7. Under Application Source, choose the Git repository type and then set the Repository URL to the clone URL of the Hello World AWS OpsWorks sample: http://github.com/awslabs/aws-flow-ruby-opsworks-helloworld.git.

    You can leave the Repository SSH Key and Branch/Revision settings blank.

    Note

    Alternatively, you can choose the Http Archive repository type, download or create a .zip or .tar.gz archive of the sample, and publish it in a accessible location (such as Amazon S3). You can then provide the public URL of the sample archive for this setting.

    For more information, see Adding Apps in the AWS OpsWorks User Guide.

  8. Under Environment Variables, add a key named AWS_REGION with its value set to the AWS region that you plan to deploy the worker to, such as us-west-2.

  9. Click Add App to complete adding the Hello World application to your layer.

Note

The helloworld.json file that exists within the repository is ignored. When AWS OpsWorks deploys the application to your Amazon EC2 instances, it will create its own JSON configuration file that contains the application settings that you have just set.

Deploy Hello World#

To see if your instance is running, click the Instances item on the left sidebar or in the Navigation menu. Once the instance is running, you can deploy the Hello World application to it.

To deploy the Hello World application to your EC2 instance:

  1. Once your instance is running, click the Apps item and click Deploy on the Hello World application that you've just finished setting up.
  2. Verify that the Command setting is Deploy, and then click the Deploy button at the bottom of the screen. The view will automatically proceed to Deployments, and you can monitor the progress of your deployment. It may take a few minutes before it is ready.

Verify Your Deployment#

Now that you have deployed Hello World to the Amazon EC2 instance, you can log in and verify your deployment. In order to do this, you will need the SSH key-pair that you associated with your Amazon EC2 instance.

To verify your deployment on the EC2 instance:

  1. If you are not already on the Instances view in your AWS OpsWorks console, select Instances in the menu or in the left sidebar.

  2. Click the SSH link. If you have not yet associated the SSH key with your account, the next screen will provide instructions and a link to do so. Otherwise, you will be presented with a list of ways to connect to your instance, through your browser or by using the command-line to SSH to the instance.

    For example, from the command-line, you can specify your private key file and the instance address, as shown in the Connect Directly section of the page you receive when clicking the SSH link:

    ssh -i ~/.ssh/YOUR-KEYFILE my-account@INSTANCE-DNS
    

    Replace YOUR-KEYFILE and INSTANCE-DNS with your key file name and the IP address of your instance, respectively.

  3. Once you are logged in to your instance, view the runner configuration file by typing the following commands (assuming that the application name you chose was helloworld):

    cd srv/www/helloworld
    cat runner_config.json
    

Important

The contents of runner_config.json must match the values that you entered when setting up your application.

Start your workflow execution#

When Hello World is deployed, AWS OpsWorks starts your workflow and activity workers, which start polling on the task lists that you specified when setting up your application.

However, since a workflow execution has not started, the workers won't receive any tasks and no activities will be run. You can start a workflow execution on the command line as was performed in Verify the Code and Configuration, or you can start the workflow execution using the AWS Management Console. Both methods will be shown.

Starting a workflow execution on the command line#

To start a workflow execution using the command line:

  1. Open a terminal (command-line) window and locate the directory where you unpacked or cloned the sample code, as per Get the Tutorial Source Code. You could also clone or unpack a new instance of the code if you don't want to modify the existing code.

  2. Whichever method you chose (whether it was to use the existing code or download a new copy), change to the tutorial code directory. For example:

    cd aws-flow-ruby-opsworks-helloworld
    
  3. Edit helloworld.json and make sure that it contains the same values that you chose when setting up your application in Add the Hello World Application. If you chose different values when setting up the application, change the values in helloworld.json to match.

    Note

    For convenience, the AWS OpsWorks version of helloworld_workflow_starter.rb uses the same helloworld.json file that the runner uses to find the domain and task lists to use. If you're curious to see how this is done, you can open the file to examine it.

  4. If you made any modifications to helloworld.json, save the file and then run hello_workflow_starter.rb just as you did when testing the code in Verify the Code and Configuration:

    bundle exec ruby lib/helloworld_workflow_starter.rb helloworld.json
    
  5. Open the AWS Management Console, navigate to the SWF section and click Workflow Executions to monitor the progress of your workflow execution. You should be able to see your workflow events in the Events tab, and your running activities in the Activities tab of the displayed workflow execution.

Starting a workflow execution using the AWS Management Console#

Alternatively to executing the workflow on the command line, you can start it using the AWS Management Console.

To start a workflow execution using the console:

  1. Using the AWS Management Console, navigate to the Amazon SWF section by selecting SWF in the Services menu.

  2. Click Dashboard and then select the same domain that you used when setting up your application in Add the Hello World Application.

  3. Click Start a new Workflow Execution in the Quick Links section of the page.

  4. Set the Workflow Type Name and Workflow Type Version that was specified in the Hello World application. You can verify these settings by viewing the helloworld_utils.rb file in the sample. If you did not modify it, these values will be HelloWorldWorkflow and 1.0. Enter any value you like for the Workflow Execution ID.

  5. Click Advanced Options and set Task List to be the same task list that you used when adding the application on AWS OpsWorks. Choose reasonable values for the Execution Start to Close Timeout and the Task Start to Close Timeout.

    For reference, the values used in the sample code are 3600 and 30 seconds, respectively.

  6. Click Continue to proceed to the next screen.

  7. Enter any values you like for the Execution Input, or leave it blank. Click the Review button to review the values that you've entered.

  8. When you're satisfied with the values, click Start Execution to start your workflow execution and begin executing your workflow and activity tasks.

  9. Click Workflow Executions and then click the name of the workflow execution you just initiated to monitor its progress as with the command-line initiated execution.

Experiment with your own application#

You've successfully created an AWS OpsWorks-ready stack, layer, instance, and application and have completed a deployment to the Amazon EC2 instance you created.

Now that you've proceeded through an entire AWS Flow Framework for Ruby deployment on AWS OpsWorks, you can try deploying your own workflow code. For the best results, follow the steps as they were presented in this guide, substituting your own application in place of the Hello World sample.

With AWS OpsWorks and the AWS Flow Framework for Ruby, you can deploy as many workflow and activity workers as you like on the AWS cloud with minimal setup!

For More Information#

For more information about working with AWS Flow Framework for Ruby and AWS OpsWorks, refer to these topics: