AWS CloudFormation
User Guide (API Version 2010-05-15)
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.Did this page help you?  Yes | No |  Tell us about it...

Automating Application Installation Using AWS CloudFormation and Cloud-Init

By Chris Whitaker, May 2011

This section shows how the Amazon Linux AMI can be used along with AWS CloudFormation to start up and configure an application dynamically at boot time. The example uses the new WaitCondition resource supported by AWS CloudFormation to wait for a Ruby on Rails application to be configured and launched before the stack is considered to be successfully created. The example also takes advantage of the Amazon Linux AMI support for Cloud-init, an open source application built by Canonical. Cloud-init enables you to use the Amazon Elastic Compute Cloud (Amazon EC2) UserData parameter to specify actions to run on your instance at boot time. The example uses this mechanism to specify the application configuration commands and the command to signal success to the wait condition so that stack creation can continue.

Note

Alternatively, you can use AWS CloudFormation helper scripts (cfn-init and cfn-signal) to automate the installation of your application. For more information, see Deploying Applications with AWS CloudFormation.

First, let’s create a simple Rails application. Start by typing the command:

$ rails new <path to application>

Before you can execute this command on an Amazon Linux instance, you must install several packages and RubyGems. Here is the full set of commands you need to create a simple application:

#!/bin/bash -ex
yum -y install gcc-c++ make
yum -y install mysql-devel sqlite-devel
yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel
gem install --no-ri --no-rdoc rails
gem install --no-ri --no-rdoc mysql
gem install --no-ri --no-rdoc sqlite3
rails new myapp
cd myapp
rails server -d

Typically, you can start a new Amazon EC2 instance running the Amazon Linux AMI from the AWS Management Console. In the console you can SSH to the instance, and type the commands listed above to set up and run a Rails application.

By using AWS CloudFormation to create your application, you can gain several advantages:

  1. To use the application you created, you need to allow access to various TCP/IP ports on your Amazon EC2 instance. In particular, you must open port 3000 so you can connect to the Rails application. You might need to open port 22 so you can SSH to your instance to manage the instance when it is up and running. To enable access to these ports, you need to create an Amazon EC2 instance, plus you need to have an appropriately configured EC2 Security Group. AWS CloudFormation lets you define an Amazon EC2 security group alongside your instance; this lets you keep your application’s entire AWS resource configuration in one place.

  2. You can use the AWS CloudFormation template to create multiple instances of your application. Each instance is guaranteed to be the same as the others. All of your application configuration and installation scripts are kept in a single place. By taking advantage of various facilities in the template, you can use the same template to create instances of your application in different Amazon EC2 Regions. For example, you can create an instance in the US-East (Northern Virginia) Region and one in the EU (Ireland) Region, safe in the knowledge that the applications are configured identically.

  3. By using the new WaitCondition resource in the AWS CloudFormation template, you know exactly when the application is ready to accept traffic.

The following code shows the full template for creating and configuring a sample Rails application in any Amazon EC2 Region.

{
 "AWSTemplateFormatVersion" : "2010-09-09",
 "Parameters" : {
    "KeyName" : {
      "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
      "Type" : "String"
    }
  },

  "Mappings" : {
    "RegionMap" : {
      "us-east-1" : {"AMI" : "ami-8c1fece5"},
      "us-west-1" : {"AMI" : "ami-3bc9997e"},
      "eu-west-1" : {"AMI" : "ami-47cefa33"},
      "ap-southeast-1" : {"AMI" : "ami-6af08e38"},
      "ap-northeast-1" : {"AMI" : "ami-300ca731"}
    }
  },

  "Resources" : {
    "Ec2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
        "KeyName" : { "Ref" : "KeyName" },
        "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
        "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
        "UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
            "#!/bin/bash -ex","\n",
            "yum -y install gcc-c++ make","\n",
            "yum -y install mysql-devel sqlite-devel","\n",
            "yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","\n",
            "gem install --no-ri --no-rdoc rails","\n",
            "gem install --no-ri --no-rdoc mysql","\n",
            "gem install --no-ri --no-rdoc sqlite3","\n",
            "rails new myapp","\n",
            "cd myapp","\n",
            "rails server -d","\n",
            "curl -X PUT -H 'Content-Type:' --data-binary '{\"Status\" : \"SUCCESS\",",
                                                           "\"Reason\" : \"The application myapp is ready\",",
                                                           "\"UniqueId\" : \"myapp\",",
                                                           "\"Data\" : \"Done\"}' ",
                  "\"", {"Ref" : "WaitForInstanceWaitHandle"},"\"\n" ]]}}
      }
    },

    "InstanceSecurityGroup" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable Access to Rails application via port 3000 and SSH access via port 22",
        "SecurityGroupIngress" : [ {
          "IpProtocol" : "tcp",
          "FromPort" : "22",
          "ToPort" : "22",
          "CidrIp" : "0.0.0.0/0"
        }, {
          "IpProtocol" : "tcp",
          "FromPort" : "3000",
          "ToPort" : "3000",
          "CidrIp" : "0.0.0.0/0"
        } ]
      }
    },

    "WaitForInstanceWaitHandle" : {
      "Type" : "AWS::CloudFormation::WaitConditionHandle",
      "Properties" : {
      }
    },

    "WaitForInstance" : {
      "Type" : "AWS::CloudFormation::WaitCondition",
      "DependsOn" : "Ec2Instance",
      "Properties" : {
        "Handle" : {"Ref" : "WaitForInstanceWaitHandle"},
        "Timeout" : "600"
      }
    }
  },

  "Outputs" : {
    "WebsiteURL" : {
      "Description" : "The URL for the newly created Rails application",
      "Value" : { "Fn::Join" : ["", [ "http://", { "Fn::GetAtt" : [ "Ec2Instance", "PublicIp" ] }, ":3000" ]]}
    }
  }
}

The template assumes that you want to have SSH access to the running Amazon EC2 instance. It requires you to enter the name of an existing Amazon EC2 key pair from your account as an input parameter when you create the stack:

"Parameters" : {
    "KeyName" : {
      "Description" : "Name of an existing EC2 KeyPair to enable SSH access to the instance",
      "Type" : "String"
    }
},

The KeyName you type in is referenced in the Amazon EC2 instance resource definition:

    "Ec2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
        "KeyName" : { "Ref" : "KeyName" },
...

If you do not already have a key pair, you can create a new one by going to the AWS Management Console and opening the Amazon EC2 console. Remember to save the key file that you create so you can use it to SSH to the instance later. For more information about creating and using Amazon EC2 key pairs, see Getting an SSH Key Pair in the Amazon EC2 User Guide.

AMI IDs are Region-specific, so the actual AMI to launch depends on the Region in which the stack is created. This template uses the “Mappings” feature to select the correct AMI based on the Region:

    "Mappings" : {
    "RegionMap" : {
      "us-east-1" : {"AMI" : "ami-8c1fece5"},
      "us-west-1" : {"AMI" : "ami-3bc9997e"},
      "eu-west-1" : {"AMI" : "ami-47cefa33"},
      "ap-southeast-1" : {"AMI" : "ami-6af08e38"},
      "ap-northeast-1" : {"AMI" : "ami-300ca731"}
    }
  },

The actual AMI to use is defined in the Amazon EC2 instance resource definition using the FindInMap intrinsic function along with the pseudo parameter AWS::Region which returns a string representing the Region in which the stack is being built:

    "Ec2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
         ...
        "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" }, "AMI" ]},
         ...
      }
    },

To open the ports for SSH access (TCP/IP port 22) and to allow access to the newly created Rails application (TCP/IP port 3000), the template defines a new security group:

    "InstanceSecurityGroup" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable Access to Rails application via port 3000 and SSH access via port 22",
        "SecurityGroupIngress" : [ {
          "IpProtocol" : "tcp",
          "FromPort" : "22",
          "ToPort" : "22",
          "CidrIp" : "0.0.0.0/0"
        }, {
          "IpProtocol" : "tcp",
          "FromPort" : "3000",
          "ToPort" : "3000",
          "CidrIp" : "0.0.0.0/0"
        } ]
      }
    },

This is referenced in the Amazon EC2 instance resource definition:

    "Ec2Instance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
          ...
        "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ],
          ...
      }
    },

Finally, to complete the application configuration, you need to configure Cloud-init to install the required packages and RubyGems and start the application. Cloud-init uses the Amazon EC2 UserData field to pass configuration information. If the UserData field starts with #!, the contents of the UserData are assumed to contain a script to execute on startup. All of the text in a CloudFormation template must conform to a JSON structure; therefore, it is necessary to escape some of the script. The sample template uses the Fn::Base64 function to base64 encode the user data and to allow parameters and references from the template to be substituted in the script at runtime (in this case a reference to the WaitConditionHandle). The example uses the Fn::Join function to concatenate the various pieces of the script.

        "UserData" : { "Fn::Base64" : { "Fn::Join" : ["",[
            "#!/bin/bash -ex","\n",
            "yum -y install gcc-c++ make","\n",
            "yum -y install mysql-devel sqlite-devel","\n",
            "yum -y install ruby-rdoc rubygems ruby-mysql ruby-devel","\n",
            "gem install --no-ri --no-rdoc rails","\n",
            "gem install --no-ri --no-rdoc mysql","\n",
            "gem install --no-ri --no-rdoc sqlite3","\n",
            "rails new myapp","\n",
            "cd myapp","\n",
            "rails server -d","\n",
            "curl -X PUT -H 'Content-Type:' --data-binary '{\"Status\" : \"SUCCESS\",",
                                                           "\"Reason\" : \"The application myapp is ready\",",
                                                           "\"UniqueId\" : \"myapp\",",
                                                           "\"Data\" : \"Done\"}' ",
                  "\"", {"Ref" : "WaitForInstanceWaitHandle"},"\"\n" ]]}}

So that the stack does not indicate CREATE_COMPLETE until the packages have been installed and the application is running, we are using the new WaitCondition resource. In the Amazon EC2 instance resource definition above, you can see that the last line in the UserData script is a CURL command that signals the WaitCondition using the WaitConditionHandle resource called WaitForInstanceWaitHandle.

The WaitCondition itself is defined as follows:

    "WaitForInstance" : {
      "Type" : "AWS::CloudFormation::WaitCondition",
      "DependsOn" : "Ec2Instance",
      "Properties" : {
        "Handle" : {"Ref" : "WaitForInstanceWaitHandle"},
        "Timeout" : "600"
      }
    }

The WaitCondition definition uses the DependsOn construct. This ensures that the WaitForInstance WaitCondition resource is only created directly after the EC2 instance resource is created. Why is this important? The Timeout value specified in the WaitCondition (in this case 600 seconds) starts ticking when the WaitCondition object is put into the CREATE_IN_PROGRESS state. In this template, we want to give the Ruby application some time to start, but not too much time (in case something bad happened with the instance). By making the WaitCondition dependent on the Amazon EC2 instance, the WaitCondition resource will only be created after the EC2 instance enters the EC2 running state and the Cloud-init script starts. Using DependsOn ensures that the configuration script has 600 seconds to run. The stack creation will fail when the WaitCondition timeout triggers if the script does not signal via the CURL command.

NOTE: The WaitCondition resource can be used to synchronize creation of other resources in the template, not just stack creation. For example, you might chose not to associate the instance with an Elastic IP address until the application is running. By adding a DependsOn clause in other resources in the template that refer to the WaitCondition, you ensure that the resources that depend on the WaitCondition are not created until the WaitCondition is signaled. Watch for more articles about passing data back from the application in the template, as well as using WaitCondition objects to wait for multiple instances to be up and running before the application is considered healthy.

If you want to download, modify, or try out the Rails sample template, it is available as part of the AWS CloudFormation sample templates.