Automate pre-workload ingestion activities for AWS Managed Services on Windows
Created by Jacob Zhang (AWS), Calvin Yeh (AWS), and Dwayne Bordelon (AWS)
Code repository: GitHub | Environment: Production | Source: Windows Servers |
Target: AWS Managed Services | R Type: Rehost | Technologies: Migration |
AWS services: AWS CloudFormation; AWS Managed Services; AWS Systems Manager; Amazon S3 |
Summary
On the Amazon Web Services (AWS) Cloud, AWS Managed Services (AMS) uses AMS workload ingest (WIGS) to move existing workloads into an AMS managed VPC. This pattern describes a solution to automate common pre-workload ingestion activities, such as upgrading .NET and Windows PowerShell and running Windows WIGS pre-ingestion validation maintained by AMS. The pattern also provides a unified user interface for the run results. It packages an AWS Systems Manager Command document, which performs the pre-ingestion activities, into an AWS CloudFormation template. The template can be deployed repeatedly without requiring access to Systems Manager itself or conflicting with automations from AMS.
Business background
Migrations to AMS require the provision of new Amazon Elastic Compute Cloud (Amazon EC2) instances using AMS managed Amazon Machine Images (AMIs) that include AMS components. Any workloads or applications running in existing data centers must be redeployed to fresh EC2 instances launched from these AMS AMIs. To avoid the potentially massive amount of manual work during the process, the AMS team built the AMS workload ingest (WIGS) workflow to onboard your custom images to AMS.
Windows instances must satisfy a few prerequisites before the WIGS process takes place. Windows PowerShell scripts are usually used to perform the necessary preparations (WIGS prep) and check if the instances are ready for WIGs (WIGS pre-ingestion validation). The prep and validation processes require an engineer to spend 15–30 minutes on each server, manually logging in and running the scripts one by one.
Business driver
Traditionally, using Systems Manager, you can automate operational tasks such as running Windows PowerShell scripts. However, because of elevated risks and frequent conflicts between automations from AMS and those from the users, AMS does not usually grant its users access to Systems Manager.
For mass migrations using AWS Application Migration Service (AWS MGN), Windows PowerShell scripts in the C:\Program Files (x86)\AWS Replication Agent\post_launch folder usually run automatically when a test or cutover instance is launched. However, these scripts, if run immediately during an instance launch, frequently conflict with automations from AMS. As a result, the launch might fail without providing the run results that you need to troubleshoot the failure.
This pattern tackles these problems and provides a working automated solution.
Prerequisites and limitations
Prerequisites
An active AWS account with AMS onboarding completed.
An Amazon Simple Storage Service (Amazon S3) bucket in the AWS account. If there is no S3 bucket over which you have control in the account, use a request for change (RFC) to create one.
The PreWIGs_CFN.json template downloaded from the ams-auto-prewigs-windows
repository. A server to which you apply this pattern must meet the following requirements:
Run Windows Server 2012 or later.
Be launched or ready to launch in the sandbox VPC migration subnet.
Have an AWS Systems Manager Agent (SSM Agent) installed.
Have an AWS Identity and Access Management (IAM) instance profile attached. The instance profile must have permissions to download files from S3 buckets in the same AWS account. An instance profile that satisfies the above-mentioned requirement is usually already established during earlier setups of a migration.
Be viewable from AWS Systems Manager Fleet Manager.
Limitations
Pre-WIGS activities vary depending on your environment and business requirements. You might need to make minor modifications to this pattern to suit your specific needs.
Product versions
The pattern is tested with Windows Server 2012, 2012 R2, 2016, and 2019. It theoretically works with later Windows versions. It does not work with earlier Windows versions.
Architecture
The architecture diagram shows the following:
A sandbox VPC with a migration subnet containing servers that have not been prepped.
The S3 bucket that stores scripts that are used by the CloudFormation template.
The CloudFormation template deploys the Systems Manager Command document. The process iterates until the steps complete.
The instances are prepped and RFCs for WIGS are made.
In the AMS managed VPC, the AMS managed subnet contains the servers after workload ingestion.
How it works
This pattern is packaged into an AWS CloudFormation template that allows infrastructure as code (IaC) repeatable deployments. You need to deploy this template only one time for each AWS account that requires this automation.
The automation is applied to all EC2 instances with a tag key AutoPreWIGs in the AWS account where this pattern is deployed. The first time an Amazon EC2 Windows instance with the tag key AutoPreWIGs starts, the automation performs the following tasks.
Upgrades Windows PowerShell to version 5.1 and .NET to version 4.5.2. The instance might reboot several times, depending on its existing Windows PowerShell and .NET versions. After each reboot, the upgrades continue until they are complete. This step uses embedded code in the CloudFormation template modified from a Windows PowerShell script
, as well as specific Systems Manager guidance on server reboots. Downloads from Amazon S3 and runs a Windows PowerShell script that you have customized to prepare the Amazon EC2 Windows instance for WIGS. For more information see the Epics section.
Installs the Windows WIGS pre-ingestion validation PowerShell module from AWS.
Runs the Windows WIGS pre-ingestion validation and makes the results viewable in Systems Manager State Manager.
Tools
AWS CloudFormation – AWS CloudFormation is a service that helps you model and set up your AWS resources. You can use a that describes all the AWS resources that you want and their dependencies, so that you can launch and configure those resources as a stack., This pattern uses a CloudFormation template to automate deployment of the resources in this pattern.
AWS Managed Services – AWS Managed Services (AMS) is an enterprise service that provides ongoing management of your AWS infrastructure. Changes made to the infrastructure in an AMS environment must be made through an RFC.
AWS Systems Manager – AWS Systems Manager (formerly known as SSM) is an AWS service that you can use to view and control your infrastructure on AWS. Using the Systems Manager console, you can view operational data from multiple AWS services and automate operational tasks across your AWS resources. This pattern uses Systems Manager to run and view the run results of the pre-WIGS activities.
Amazon S3 – Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This pattern uses Amazon S3 to store the CloudFormation template and a Windows PowerShell script that is downloaded.
Epics
| Task | Description | Skills required |
|---|---|---|
| Perform necessary changes to the servers based on business needs. | If you need changes automatically applied to your servers before their ingestions, create a Windows PowerShell script named Important: The script must not contain instructions to reboot the server, and it must not require administrator privileges. | PowerShell scripting |
| Remove software that isn’t supported by AMS. | AMS requires certain software, such as antivirus applications and VMware Tools removed before WIGS runs. Include the uninstallation in the | PowerShell scripting |
| Task | Description | Skills required |
|---|---|---|
| Create a folder in S3. | In an S3 bucket in the same AWS account where you deploy this pattern, create a folder. | General AWS |
| Upload the scripts. | Upload the | General AWS |
| Task | Description | Skills required |
|---|---|---|
| Select the change type. | Navigate to the AMS console to create an RFC. Use the Create Stack from CloudFormation (CFN) Template change type. | General AMS |
| Set run parameters for the path to the CloudFormation template. | In the Execution configuration section, expand Additional configuration. In the CloudFormation template S3 endpoint box, paste the URL to the CloudFormation template. | General AMS |
| Specify the path to the Amazon S3 folder. | Under Parameters, use ScriptSource as the Name. For Value, enter the path to the S3 folder that contains the Windows PowerShell scripts. Make sure that you use the | General AMS |
| Deploy the stack. | To deploy the stack, choose Create. | General AMS |
| Escalate the RFC to AMS Ops. | The RFC must be implemented manually by the AMS Ops team because it uses Systems Manager to deploy resources with and requires a security review. As soon as you create the RFC, it will be automatically rejected by the system. Choose the RFC, and add a correspondence to the RFC stating Please execute manually. Note the RFC ID, and escalate it with a service request. | General AMS |
| Task | Description | Skills required |
|---|---|---|
| Add the AutoPreWIGs tag to instances. | Note IDs of all instances to which you want to apply this automation and wait at least 30 minutes for the instance to finish the automations implemented by AMS. Submit an automated RFC to add the tag with AutoPreWIGs as the key and any string, such as 1, as the value. The automation will be applied a few minutes after you add the tag. | General AMS |
| Verify the automation results. | Open the Systems Manager console, and choose State Manager. Choose the Association ID with the name AMS-PreWIG-Prep-and-Validation-Association. On the Execution history tab, you can see the results of the automation. | General AMS |
| Fix any errors. | If the automation fails, choose its Execution ID. You can see the run results for each EC2 instance. To see the details for each step of the automation, choose Output. If a particular step fails, use the information in the Output and the Error sections to diagnose the problem. | Migration engineer |
| Remove the AutoPreWIGs tag. | Important: After you fix the errors, if any, submit an automated RFC to remove the AutoPreWIGs tag. WIGS will fail if you don't remove the tag. | General AMS |
| Task | Description | Skills required |
|---|---|---|
| Submit RFCs for WIGS. | Now that the instances are ready for workload ingestion, submit the RFCs for WIGS. | General AMS |
Related resources
