Automated Deployment - Scale-Out Computing on AWS

Automated Deployment

Before you launch the automated deployment, please review the architecture, prerequisites, and other considerations discussed in this guide. Follow the step-by-step instructions in this section to configure and deploy the Scale-Out Computing on AWS solution into your account.

Time to deploy: Approximately 35 minutes

Prerequisites

Subscribe to Amazon Machine Images

This solution uses an Amazon Machine Image (AMI) as the host operating system for the scheduler instance, user desktop instances, and compute node instances. By default, you must select the base AMI to use for all three instances in the Linux Distribution template parameter or specify a Custom AMI. As of this release, this solution supports the following AMIs for the scheduler instance:

  • Red Hat Enterprise Linux 7

  • CentOS 7

  • Amazon Linux 2

Note

If you choose to use the CentOS 7 image, you must subscribe to CentOS 7 in the AWS Marketplace, to allow the installer to access the AMI during installation.

This solution supports a heterogeneous environment. After installation, administrators and users can specify a custom AMI per job and queue.

Service-Linked Roles for Amazon EC2 Auto Scaling

This solution deploys Amazon EC2 Auto Scaling to scale out multi-instance, user-submitted jobs. Verify that AWS Identity and Access Management (AWS IAM) roles have the appropriate permissions supporting EC2 Autoscaling. For more information, see Auto Scaling Service-Linked Roles.

Select Your Deployment

This solution can be deployed using a default set of parameters in the AWS CloudFormation template, or you can customize the solution by building your own custom installer by cloning the GitHub repository.

Deploying the AWS CloudFormation template with the default parameters is recommended for testing and proof of concept. However, if you are using this solution in a production environment, we recommend deploying this solution using a custom installer in your own hosted repository to reduce costs, maintain customization and extensibility. If you choose to deploy this solution using a custom installer, see Appendix A.

What We'll Cover

The procedure for deploying this architecture on AWS consists of the following steps. For detailed instructions, follow the links for each step.

Step 1. Launch the Stack

  • Launch the AWS CloudFormation template into your AWS account.

  • Enter values for required parameter: Stack Name, User Name, Password

  • Review the other template parameters, and adjust if necessary.

Step 2. Access the Amazon EC2 Cluster

  • Access the Amazon EC2 cluster through the UI or SSH.

Step 3. Manage Your Budget and Monitor Costs

  • Set up cost allocation and budgets

Step 1. Launch the Stack

This automated AWS CloudFormation template deploys Scale-Out Computing on AWS in the AWS Cloud. Verify that you have completed the prerequisites before you launch the stack.

Note

You are responsible for the cost of the AWS services used while running this solution. See the Cost section for more details. For full details, see the pricing webpage for each AWS service you will be using in this solution.

  1. Sign in to the AWS Management Console and click the button below to launch the scale-out-computing-on-aws AWS CloudFormation template.

    
                                Scale-Out Computing on AWS launch button

    You can also download the template as a starting point for your own implementation.

  2. The template launches in the US East (N. Virginia) Region by default. To launch this solution in a different AWS Region, use the Region selector in the console navigation bar. Note that if you choose to launch in a different Region, we recommend using a custom installer in your own Amazon S3 bucket to avoid data transfer costs. For more information, see Appendix A.

    Note

    This solution uses three Availability Zones to maximize the resources users have for submitting jobs. Therefore, you must launch this solution in an AWS Region that has at least three Availability Zones. For more information, see Global Infrastructure.

  3. On the Create stack page, verify that the correct template URL shows in the Amazon S3 URL text box and choose Next.

  4. On the Specify stack details page, assign a name to your solution stack.

    Note

    This solution deploys an analytics stack that limits your stack name to 24 lowercase characters. The template automatically adds the prefix soca- to your solution stack name.

  5. Under Parameters, review the parameters for the template and modify them as necessary. This solution uses the following default values.

    Parameter Default Description
    Install Location
    Installer S3 Bucket solutions-reference

    The default AWS bucket name. Do not change this parameter unless you are using a custom installer.

    Installer Folder scale-out-computing-on-aws/latest/

    The default AWS folder name. Do not change this parameter unless you are using a custom installer.

    Linux Distribution
    Linux Distribution AmazonLinux2

    Select the preferred Linux distribution for the scheduler and compute instances.

    Custom AMI <Optional input>

    If using a customized Amazon Machine Image, enter the ID.

    Note

    If you are using your own AMI, you still have to specify the base Linux Operating System. For more information, see Subscribe to Amazon Machine Images.

    Network and Security
    EC2 Instance Type for Scheduler m5.large Select the instance type for the scheduler.
    VPC CLuster CIDR 10.0.0.0/16 Choose the CIDR (/16) block for the VPC. This is the internal network over which your cluster will communicate.
    IP Address <Requires input> Identifies the default IP(s) allowed to directly SSH into the scheduler and access Amazon ES. To restrict access, use IP/subnet (x.x.x.x/32 for your own IP or x.x.x.x/24 for a range. Replace x.x.x.x with the PUBLIC IP. To identify the public IP, use a tool, such as https://ifconfig.co/.
    Note

    0.0.0.0/0 means ALL INTERNET access and is not recommended.

    Key Pair Name <Requires input> Public/private key pair, which allows you to connect securely to your instance after it launches. When you created an AWS account, this is the key pair you created in your preferred region.
    Default LDAP user
    User Name <Requires input> The username for your default LDAP user.
    Password <Requires input> The password for your default LDAP user.
    Note

    5 characters minimum. Must start with a letter. Accepted special characters include: ! # @ + _ ^ { } [ ] ( )

  6. Choose Next.

  7. On the Configure Stack Options page, choose Next. Optionally, you can specify tags to associate with the entire stack and specify an IAM role that will be used for all of the stack creation functions.

  8. On the Review page, review and confirm the settings. Be sure to check the box acknowledging that the template will create AWS Identity and Access Management (IAM) resources.

  9. Choose Create stack to deploy the stack.

    You can view the status of the stack in the AWS CloudFormation console in the Status column. You should see a status of CREATE_COMPLETE in approximately 35 minutes. Note that the custom installer deployment takes approximately 60 minutes.

Step 2. Access the Amazon EC2 Cluster

After the AWS CloudFormation template has successfully launched you can access the Amazon EC2 cluster via HTTPS through a web portal or SSH.

Access the Cluster Through the Web Portal

Use the following procedure to access the cluster through the web portal:

  1. After the solution has deployed, navigate to the stack Outputs tab

  2. Copy the value for WebUserInterface, and paste the link into a web browser.

Note

It can take up to 15 minutes for the UI to be installed after the AWS CloudFormation template is completed.

To open the webpage, you must authorize the web browser to trust the self-signed security certificate (add an exception) or Upload a Server Certificate to the Elastic Load Balancer endpoint. Note that access to the web UI will be restricted to the subnet specified in the IP Address template parameter.

Access the Cluster Through SSH

Administrator SSH

By default, this solution installs the key pair that you provided in the Key Pair Name template parameter. Note that all SSH sessions are required to use public key encryption.

Additionally, this solution provides an admin connection string for operating systems with native SSH clients (Mac/Linux) in the Outputs section of the CloudFormation stack.

Note

It can take up to 15 minutes for final installation procedures to complete. During this time, attempts to login via SSH will generate a message indicating that the installation is still running.

User SSH

Users who access the cluster through SSH must download their PEM key. For more information on downloading your PEM/PPK key and setting up the SSH client, see How to access the solution.

Adding Users to the Cluster

Scale-Out Computing on AWS uses open LDAP for directory services. Administrators can interact with their directory using LDAP directly. This solution installs a custom wrapper that can be found in /apps/soca/cluster_manager/ldap_manager.py. This wrapper can be run by administrators logged in via SSH, or for quick actions via the web portal. For example, adding/deleting users, resetting user passwords, and granting and revoking administrator privileges.

Use the following procedure to create or delete a user through the user interface:

  1. In Admin, navigate to Users Management.

  2. Add and delete users.

Note

Deleting users will prevent user access to the cluster but will not remove associated $HOME directory and data.

Step 3. Manage Your Budget and Monitor Costs

By default, this solution implements comprehensive tagging of cluster resources and enables admin-defined tags during cluster deployment. All Amazon Elastic Compute Cloud (Amazon EC2) resources launched by this solution come with Amazon EC2 tags that can be used to get detailed information about your cluster usage. You can modify and add tags based on your business needs.


                Scale-Out Computing on AWS default tags

Figure 2: Scale-Out Computing on AWS default tags

AWS Cost Explorer

Use the following procedures to setup cost allocation and budgets to track the costs associated with solution resources running in your account.

Enable Cost Allocation Tags

  1. In the AWS Cost Management console, select your account name, then select My Billing Dashboard

  2. In the left-hand navigation pane, select Cost allocation tags

  3. Search all tags, then select Activate

    Note

    Tags may take up to 24 hours to activate.

Enable Cost Explorer

  1. In the AWS Cost Management console, select My Billing Dashboard.

  2. Select Cost Explorer, the seclect Enable Cost Explorer.

    Note

    Cost Explorer may take up to 24 hours to be enabled.

Query Cost Explorer

  1. Select Cost Explorer then specify your filters.

    The following example shows the EC2-Instance cost group by day for the queue named cpus:

    
                        EC2-Instance cost group by day

    Figure 3: EC2-Instance cost group by day

  2. For more detailed information, select Group By and apply any additional filters.

  3. Select Tag and select the tag soca:JobOwner. The graph will automatically be updated with a cost breakdown by users for the cpus queue.

    The following example shows user level information for the cpus queue.

    
                            User level information for cpus queue

    Figure 4: User level information for cpus queue

AWS Budgets

  1. In the AWS Budgets console, select your Billing Dashboard.

  2. In the navigation pane, select Budget, then select Create a budget.

  3. Specify your Cost Budget, then apply the tag(s) you want to monitor (i.e. cost center, team, user, and queue or project), and the budget limit you do not want to exceed.