Deploy an environment for containerized Blu Age applications by using Terraform

Created by Richard Milner-Watts (AWS)

Code repository: Blu Age Sample ECS Infrastructure (Terraform)	Environment: Production	Source: Mainframe
Target: Containers	R Type: Replatform	Workload: IBM; All other workloads
Technologies: Mainframe; Containers & microservices	AWS services: Amazon ECS; AWS Step Functions; Amazon VPC; Amazon Aurora

Summary

Migrating legacy mainframe workloads into modern cloud architectures can eliminate the costs of maintaining a mainframe—costs that only increase as the environment ages. However, migrating jobs from a mainframe can pose unique challenges. Internal resources might not be familiar with the job logic, and the high performance of mainframes at these specialized tasks can be difficult to replicate when compared to commodity, generalized CPUs. Rewriting these jobs can be a large undertaking and require significant effort.

Blu Age converts legacy mainframe workloads into modern Java code, which you can then run as a container.

This pattern provides a sample serverless architecture for running a containerized application that has been modernized with the Blu Age tool. The included HashiCorp Terraform files will build a secure architecture for the orchestration of Blu Age containers, supporting both batch tasks and real-time services.

For more information about modernizing your workloads by using Blu Age and AWS services, see these AWS Prescriptive Guidance publications:

• Running mainframe workloads that have been modernized with Blu Age on AWS serverless infrastructure

• Containerize mainframe workloads that have been modernized by Blu Age

For assistance with using Blu Age to modernize your mainframe workloads, contact the Blu Age team by choosing Contact our experts on the Blu Age website. For assistance with migrating your modernized workloads to AWS, integrating them with AWS services, and moving them into production, contact your AWS account manager or fill out the AWS Professional Services form.

Prerequisites and limitations

Prerequisites

• The sample containerized Blu Age application provided by the Containerize mainframe workloads that have been modernized by Blu Age pattern. The sample application provides the logic to handle the processing of input and output for the modernized application, and it can integrate with this architecture.

• Terraform is required to deploy these resources.

Limitations

• Amazon Elastic Container Service (Amazon ECS) places limits on the task resources that can be made available to the container. These resources include CPU, RAM, and storage. For example, when using Amazon ECS with AWS Fargate, the task resource limits apply.

Product versions

This solution was tested with the following versions:

• Terraform 1.3.6

• Terraform AWS Provider 4.46.0

Architecture

Source technology stack

• Blu Age

• Terraform

Target technology stack

• Amazon Aurora PostgreSQL-Compatible Edition

• AWS Backup

• Amazon Elastic Container Registry (Amazon ECR)

• Amazon ECS

• AWS Identity and Access Management Service (IAM)

• AWS Key Management Server (AWS KMS)

• AWS Secrets Manager

• Amazon Simple Notification Service (Amazon SNS)

• Amazon Simple Storage Service (Amazon S3)

• AWS Step Functions

• AWS Systems Manager

Target architecture

The following diagram shows the solution architecture.

1. The solution deploys the following IAM roles:

   • Batch task role

   • Batch task execution role

   • Service task role

   • Service task execution role

   • Step Functions role

   • AWS Backup role

   • RDS Enhanced Monitoring role.

   The roles conform to least-privileged access principles.

2. Amazon ECR is used to store the container image that is orchestrated by this pattern.

3. AWS Systems Manager Parameter Store provides configuration data about each environment to the Amazon ECS task definition at runtime.

4. AWS Secrets Manager provides sensitive configuration data about the environment to the Amazon ECS task definition at runtime. The data has been encrypted by AWS KMS.

5. The Terraform modules create Amazon ECS task definitions for all real-time and batch tasks.

6. Amazon ECS runs a batch task by using AWS Fargate as the compute engine. This is a short-lived task, initiated as required by AWS Step Functions.

7. Amazon Aurora PostgreSQL-Compatible provides a database to support the modernized application. This replaces mainframe databases such as IBM Db2 or IBM IMS DB.

8. Amazon ECS runs a long-lived service to deliver a modernized real-time workload. These stateless applications run permanently with containers spread across Availability Zones.

9. A Network Load Balancer is used to grant access to the real-time workload. The Network Load Balancer supports earlier protocols, such as IBM CICS. Alternatively, you can use an Application Load Balancer with HTTP-based workloads.

10. Amazon S3 provides object storage for job inputs and outputs. The container should handle pull and push operations into Amazon S3 to prepare the working directory for the Blu Age application.

11. The AWS Step Functions service is used to orchestrate running the Amazon ECS tasks to process batch workloads.

12. SNS topics for each batch workload are used to integrate the modernized application with other systems, such as email, or to initiate additional actions, such as delivering output objects from Amazon S3 into FTP.

Note: By default, the solution has no access to the internet. This pattern assumes that the virtual private cloud (VPC) will be connected to other networks using a service such as AWS Transit Gateway. As such, multiple interface VPC endpoints are deployed to grant access to the AWS services used by the solution. To turn on direct internet access, you can use the toggle in the Terraform module to replace the VPC endpoints with an internet gateway and the associated resources.

Automation and scale

The use of serverless resources throughout this pattern helps to ensure that, by scaling out, there are few limits on the scale of this design. This reduces noisy neighbor concerns, such as the competition for compute resources that might be experienced on the original mainframe. Batch tasks can be scheduled to run simultaneously as needed.

Individual containers are limited by the maximum sizes supported by Fargate. For more information, see the Task CPU and memory section in the Amazon ECS documentation.

To scale real-time workloads horizontally, you can add containers.

Tools

AWS services

• Amazon Aurora PostgreSQL-Compatible Edition is a fully managed, ACID-compliant relational database engine that helps you set up, operate, and scale PostgreSQL deployments.

• AWS Backup is a fully managed service that helps you centralize and automate data protection across AWS services, in the cloud, and on premises.

• Amazon Elastic Container Registry (Amazon ECR) is a managed container image registry service that’s secure, scalable, and reliable.

• Amazon Elastic Container Service (Amazon ECS) is a fast and scalable container management service that helps you run, stop, and manage containers on a cluster.

• AWS Identity and Access Management (IAM) helps you securely manage access to your AWS resources by controlling who is authenticated and authorized to use them.

• AWS Key Management Service (AWS KMS) helps you create and control cryptographic keys to help protect your data.

• AWS Secrets Manager helps you replace hardcoded credentials in your code, including passwords, with an API call to Secrets Manager to retrieve the secret programmatically.

• Amazon Simple Notification Service (Amazon SNS) helps you coordinate and manage the exchange of messages between publishers and clients, including web servers and email addresses.

• Amazon Simple Storage Service (Amazon S3) is a cloud-based object storage service that helps you store, protect, and retrieve any amount of data.

• AWS Step Functions is a serverless orchestration service that helps you combine AWS Lambda functions and other AWS services to build business-critical applications.

• AWS Systems Manager Parameter Store provides secure, hierarchical storage for configuration data management and secrets management.

Other services

• HashiCorp Terraform is an open-source infrastructure as code (IaC) tool that helps you use code to provision and manage cloud infrastructure and resources. This pattern uses Terraform to create the sample architecture.

Code repository

The source code for this pattern is available in the GitHub Blu Age Sample ECS Infrastructure (Terraform) repository.

Best practices

• For test environments, use features such as the forceDate option to configure the modernized application to generate consistent test results by always running for a known time period.

• Tune each task individually to consume the optimal amount of resources. You can use Amazon CloudWatch Container Insights to obtain guidance on potential bottlenecks.

Epics

Prepare the environment for deployment

Task	Description	Skills required
Clone the solution source code.	Clone the solution code from the GitHub project.	DevOps engineer
Bootstrap the environment by deploying resources to store the Terraform state.	Open a terminal window, and confirm that Terraform is installed and that AWS credentials are available. Navigate to the bootstrap-terraform folder. Edit the file main.tf if you want to change the names of the S3 bucket (<accountId>-terraform-backend) and Amazon DynamoDB table (terraform-lock). Run the terraform apply command to deploy the resources. Make a note of the S3 bucket and DynamoDB table names.	DevOps engineer

Deploy the solution infrastructure

Task	Description	Skills required
Review and update the Terraform configuration.	In the root directory, open the file main.tf, review the contents, and consider making the following updates: Update the AWS Region by searching for and replacing the string eu-west-1 with desired Region that you want to use. Update the bucket name in the Terraform Backend block if the default was altered in the previous epic. Update the dynamodb_table value if the default was altered in the previous epic. Update the value of the stack_prefix variable to the string that you want. This string will be prepended to the names of all resources created by this pattern. Update the value of the vpc_cidr This should be at least a /24 address range. Review the Locals section. This is used to define the Blu Age tasks that will be deployed. The solution will iterate over the list object bluage_batch_modules, creating the associated resources (Step Functions state machine, task definition, and SNS topic) for each element of the list. In some cases, you might want to adjust variables for different environments. For example, to force the runtime in test environments, you can change value of the force_execution_time variable. To turn on internet access, change the value for direct_internet_access_required from false to true. This will deploy an internet gateway, along with the NAT gateways and route tables that turn on public internet access for the infrastructure. By default, the solution will deploy interface VPC endpoints into a VPC without direct internet access. To grant access to any client-server workloads that are served through Elastic Load Balancing, update the values of additional_nlb_igress_cidrs with the CIDR networks that should be allowed.	DevOps engineer
Deploy the Terraform file.	From your terminal, run the terraform apply command to deploy all resources. Review the changes generated by Terraform, and enter yes to initiate the build. Note that it can take over 15 minutes to deploy this infrastructure.	DevOps engineer

(Optional) Deploy a valid Blu Age containerized application

Task	Description	Skills required
Push the Blu Age container image to Amazon ECR.	Push the container into the Amazon ECR repository that you created in the previous epic. For instructions, see the Amazon ECR documentation. Make a note of the container image URI.	DevOps engineer
Update the Terraform to reference the Blu Age container image.	Update the file main.tf to reference the container image that you uploaded.	DevOps engineer
Redeploy the Terraform file.	From your terminal, run terraform apply to deploy all resources. Review the suggested updates from Terraform, and then enter yes to proceed with the deployment.	DevOps engineer

Related resources

• Blu Age

• Running mainframe workloads that have been modernized with Blu Age on AWS serverless infrastructure

• Containerize mainframe workloads that have been modernized by Blu Age