Using Amazon ECS Windows containers with domainless gMSA using the AWS CLI
The following tutorial shows how to create an Amazon ECS task that runs a Windows container that has credentials to access Active Directory with the AWS CLI. By using domainless gMSA, the container instance isn't joined to the domain, other applications on the instance can't use the credentials to access the domain, and tasks that join different domains can run on the same instance.
Topics
- Prerequisites
- Step 1: Create and configure the gMSA account on Active Directory Domain Services (AD DS)
- Step 2: Upload Credentials to Secrets Manager
- Step 3: Modify your CredSpec JSON to include domainless gMSA information
- Step 4: Upload CredSpec to Amazon S3
- Step 5: (Optional) Create an Amazon ECS cluster
- Step 6: Create an IAM role for container instances
- Step 7: Create a custom task execution role
- Step 8: Create a task role for Amazon ECS Exec
- Step 9: Register a task definition that uses domainless gMSA
- Step 10: Register a Windows container instance to the cluster
- Step 11: Verify the container instance
- Step 12: Run a Windows task
- Step 13: Verify the container has gMSA credentials
- Step 14: Clean up
- Debugging Amazon ECS domainless gMSA for Windows containers
Prerequisites
This tutorial assumes that the following prerequisites have been completed:
-
The steps in Set up to use Amazon ECS have been completed.
-
Your AWS user has the required permissions specified in the AmazonECS_FullAccess IAM policy example.
-
The latest version of the AWS CLI is installed and configured. For more information about installing or upgrading your AWS CLI, see Installing the AWS Command Line Interface.
-
You set up an Active Directory domain with the resources that you want your containers to access. Amazon ECS supports the following setups:
-
An AWS Directory Service Active Directory. AWS Directory Service is an AWS managed Active Directory that's hosted on Amazon EC2. For more information, see Getting Started with AWS Managed Microsoft AD in the AWS Directory Service Administration Guide.
-
An on-premises Active Directory. You must ensure that the Amazon ECS Linux container instance can join the domain. For more information, see AWS Direct Connect.
-
-
You have a VPC and subnets that can resolve the Active Directory domain name.
-
You chose between domainless gMSA and joining each instance to a single domain. By using domainless gMSA, the container instance isn't joined to the domain, other applications on the instance can't use the credentials to access the domain, and tasks that join different domains can run on the same instance.
Then, choose the data storage for the CredSpec and optionally, for the Active Directory user credentials for domainless gMSA.
Amazon ECS uses an Active Directory credential specification file (CredSpec). This file contains the gMSA metadata that's used to propagate the gMSA account context to the container. You generate the CredSpec file and then store it in one of the CredSpec storage options in the following table, specific to the Operating System of the container instances. To use the domainless method, an optional section in the CredSpec file can specify credentials in one of the domainless user credentials storage options in the following table, specific to the Operating System of the container instances.
Storage location Linux Windows Amazon Simple Storage Service CredSpec CredSpec AWS Secrets Manager domainless user credentials domainless user credentials Amazon EC2 Systems Manager Parameter Store CredSpec CredSpec, domainless user credentials Local file N/A CredSpec -
(Optional) AWS CloudShell is a tool that gives customers a command line without needing to create their own EC2 instance. For more information, see What is AWS CloudShell? in the AWS CloudShell User Guide.
Step 1: Create and configure the gMSA account on Active Directory Domain Services (AD DS)
Create and configure a gMSA account on the Active Directory domain.
-
Generate a Key Distribution Service root key
Note
If you are using AWS Directory Service, then you can skip this step.
The KDS root key and gMSA permissions are configured with your AWS managed Microsoft AD.
If you have not already created a gMSA Service Account in your domain, you’ll need to first generate a Key Distribution Service (KDS) root key. The KDS is responsible for creating, rotating, and releasing the gMSA password to authorized hosts. When the
ccg.exe
needs to retrieve gMSA credentials, it contact KDS to retrieve the current password.To check if the KDS root key has already been created, run the following PowerShell cmdlet with domain admin privileges on a domain controller using the
ActiveDirectory
PowerShell module. For more information about the module, see ActiveDirectory Moduleon the Microsoft Learn website. PS C:\>
Get-KdsRootKey
If the command returns a key ID, you can skip the rest of this step. Otherwise, create the KDS root key by running the following command:
PS C:\>
Add-KdsRootKey -EffectiveImmediately
Although the argument
EffectiveImmediately
to the command implies the key is effective immediately, you need to wait 10 hours before the KDS root key is replicated and available for use on all domain controllers. -
Create the gMSA account
To create the gMSA account and allow the
ccg.exe
to retrieve the gMSA password, run the following PowerShell commands from a Windows Server or client with access to the domain. ReplaceExampleAccount
with the name that you want for your gMSA account.-
PS C:\>
Install-WindowsFeature RSAT-AD-PowerShell
-
PS C:\>
New-ADGroup -Name "
ExampleAccount
Authorized Hosts" -SamAccountName "ExampleAccount
Hosts" -GroupScope DomainLocal -
PS C:\>
New-ADServiceAccount -Name "
ExampleAccount
" -DnsHostName "contoso
" -ServicePrincipalNames "host/ExampleAccount
", "host/contoso
" -PrincipalsAllowedToRetrieveManagedPassword "ExampleAccount
Hosts" -
Create a user with a permanent password that doesn't expire. These credentials are stored in AWS Secrets Manager and used by each task to join the domain.
PS C:\>
New-ADUser -Name "
ExampleAccount
" -AccountPassword (ConvertTo-SecureString -AsPlainText "Test123
" -Force) -Enabled 1 -PasswordNeverExpires 1 -
PS C:\>
Add-ADGroupMember -Identity "
ExampleAccount
Hosts" -Members "ExampleAccount
" -
Install the PowerShell module for creating CredSpec objects in Active Directory and output the CredSpec JSON.
PS C:\>
Install-PackageProvider -Name NuGet -Force
PS C:\>
Install-Module CredentialSpec
-
PS C:\>
New-CredentialSpec -AccountName
ExampleAccount
-
-
Copy the JSON output from the previous command into a file called
gmsa-cred-spec.json
. This is the CredSpec file. It is used in Step 3, Step 3: Modify your CredSpec JSON to include domainless gMSA information.
Step 2: Upload Credentials to Secrets Manager
Copy the Active Directory credentials into a secure credential storage system, so that each task retrieves it. This is the domainless gMSA method. By using domainless gMSA, the container instance isn't joined to the domain, other applications on the instance can't use the credentials to access the domain, and tasks that join different domains can run on the same instance.
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
Run the following AWS CLI command and replace the username, password, and domain name to match your environment. Keep the ARN of the secret to use in the next step, Step 3: Modify your CredSpec JSON to include domainless gMSA information
The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws secretsmanager create-secret \ --name
gmsa-plugin-input
\ --description "Amazon ECS - gMSA Portable Identity.
" \ --secret-string "{\"username\":\"ExampleAccount
\",\"password\":\"Test123
\",\"domainName\":\"contoso.com
\"}"
Step 3: Modify your CredSpec JSON to include domainless gMSA information
Before uploading the CredSpec to one of the storage options, add information to the
CredSpec with the ARN of the secret in Secrets Manager from the previous step. For more information,
see Additional credential spec configuration for non-domain-joined container host use
case
-
Add the following information to the CredSpec file inside the
ActiveDirectoryConfig
. Replace the ARN with the secret in Secrets Manager from the previous step.Note that the
PluginGUID
value must match the GUID in the following example snippet and is required."HostAccountConfig": { "PortableCcgVersion": "1", "PluginGUID": "{859E1386-BDB4-49E8-85C7-3070B13920E1}", "PluginInput": "{\"credentialArn\": \"arn:aws:secretsmanager:
aws-region
:111122223333:secret:gmsa-plugin-input
\"}" }You can also use a secret in SSM Parameter Store by using the ARN in this format:
\"arn:aws:ssm:
.aws-region
:111122223333:parameter/gmsa-plugin-input
\" -
After you modify the CredSpec file, it should look like the following example:
{ "CmsPlugins": [ "ActiveDirectory" ], "DomainJoinConfig": { "Sid": "S-1-5-21-4066351383-705263209-1606769140", "MachineAccountName": "ExampleAccount", "Guid": "ac822f13-583e-49f7-aa7b-284f9a8c97b6", "DnsTreeName": "contoso", "DnsName": "contoso", "NetBiosName": "contoso" }, "ActiveDirectoryConfig": { "GroupManagedServiceAccounts": [ { "Name": "ExampleAccount", "Scope": "contoso" }, { "Name": "ExampleAccount", "Scope": "contoso" } ], "HostAccountConfig": { "PortableCcgVersion": "1", "PluginGUID": "{859E1386-BDB4-49E8-85C7-3070B13920E1}", "PluginInput": "{\"credentialArn\": \"arn:aws:secretsmanager:
aws-region
:111122223333:secret:gmsa-plugin-input
\"}" } } }
Step 4: Upload CredSpec to Amazon S3
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
Copy the CredSpec file to the computer or environment that you are running AWS CLI commands in.
-
Run the following AWS CLI command to upload the CredSpec to Amazon S3. Replace
MyBucket
with the name of your Amazon S3 bucket. You can store the file as an object in any bucket and location, but you must allow access to that bucket and location in the policy that you attach to the task execution role.The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws s3 cp gmsa-cred-spec.json \ s3://
MyBucket/ecs-domainless-gmsa-credspec
Step 5: (Optional) Create an Amazon ECS cluster
By default, your account has an Amazon ECS cluster named default
. This cluster is
used by default in the AWS CLI, SDKs, and AWS CloudFormation. You can use additional clusters to group and
organize tasks and infrastructure, and assign defaults for some configuration.
You can create a cluster from the AWS Management Console, AWS CLI, SDKs, or AWS CloudFormation. The settings and configuration in the cluster don't affect gMSA.
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
$
aws ecs create-cluster --cluster-name windows-domainless-gmsa-cluster
Important
If you choose to create your own cluster, you must specify --cluster
clusterName
for each command that you intend to use with that cluster.
Step 6: Create an IAM role for container instances
A container instance is a host computer to run containers in ECS tasks, for example Amazon EC2 instances. Each container instance registers to an Amazon ECS cluster. Before you launch Amazon EC2 instances and register them to a cluster, you must create an IAM role for your container instances to use.
To create the container instance role, see Amazon ECS container instance IAM role. The default ecsInstanceRole
has sufficient
permissions to complete this tutorial.
Step 7: Create a custom task execution role
Amazon ECS can use a different IAM role for the permissions needed to start each task, instead of the container instance role. This role is the task execution role. We recommend creating a task execution role with only the permissions required for ECS to run the task, also known as least-privilege permissions. For more information about the principle of least privilege, see SEC03-BP02 Grant least privilege access in the AWS Well-Architected Framework.
-
To create a task execution role, see Creating the task execution role. The default permissions allow the container instance to pull container images from Amazon Elastic Container Registry and
stdout
andstderr
from your applications to be logged to Amazon CloudWatch Logs.Because the role needs custom permissions for this tutorial, you can give the role a different name than
ecsTaskExecutionRole
. This tutorial usesecsTaskExecutionRole
in further steps. -
Add the following permissions by creating a custom policy, either an inline policy that only exists in for this role, or a policy that you can reuse. Replace the ARN for the
Resource
in the first statement with the Amazon S3 bucket and location, and the secondResource
with the ARN of the secret in Secrets Manager.If you encrypt the secret in Secrets Manager with a custom key, you must also allow
kms:Decrypt
for the key.If you use SSM Parameter Store instead of Secrets Manager, you must allow
ssm:GetParameter
for the parameter, instead ofsecretsmanager:GetSecretValue
.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": "arn:aws:s3:::
MyBucket/ecs-domainless-gmsa-credspec/gmsa-cred-spec.json
" }, { "Effect": "Allow", "Action": [ "secretsmanager:GetSecretValue" ], "Resource": "arn:aws:secretsmanager:aws-region
:111122223333:secret:gmsa-plugin-input
" } ] }
Step 8: Create a task role for Amazon ECS Exec
This tutorial uses Amazon ECS Exec to verify functionality by running a
command inside a running task. To use ECS Exec, the service or task must turn on ECS Exec and the
task role (but not the task execution role) must have ssmmessages
permissions. For
the required IAM policy, see ECS Exec permissions.
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
To create a task role using the AWS CLI, follow these steps.
-
Create a file called
ecs-tasks-trust-policy.json
with the following contents:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ecs-tasks.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
-
Create an IAM role. You can replace the name
ecs-exec-demo-task-role
but keep the name for following steps.The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws iam create-role --role-name
ecs-exec-demo-task-role
\ --assume-role-policy-document file://ecs-tasks-trust-policy.jsonYou can delete the file
ecs-tasks-trust-policy.json
. -
Create a file called
ecs-exec-demo-task-role-policy.json
with the following contents:{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ssmmessages:CreateControlChannel", "ssmmessages:CreateDataChannel", "ssmmessages:OpenControlChannel", "ssmmessages:OpenDataChannel" ], "Resource": "*" } ] }
-
Create an IAM policy and attach it to the role from the previous step.
The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws iam put-role-policy \ --role-name
ecs-exec-demo-task-role
\ --policy-name ecs-exec-demo-task-role-policy \ --policy-document file://ecs-exec-demo-task-role-policy.jsonYou can delete the file
ecs-exec-demo-task-role-policy.json
.
Step 9: Register a task definition that uses domainless gMSA
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
Create a file called
windows-gmsa-domainless-task-def.json
with the following contents:{ "family": "windows-gmsa-domainless-task", "containerDefinitions": [ { "name": "windows_sample_app", "image": "mcr.microsoft.com/windows/servercore/iis", "cpu": 1024, "memory": 1024, "essential": true, "credentialSpecs": [ "credentialspecdomainless:arn:aws:s3:::ecs-domainless-gmsa-credspec/gmsa-cred-spec.json" ], "entryPoint": [ "powershell", "-Command" ], "command": [ "New-Item -Path C:\\inetpub\\wwwroot\\index.html -ItemType file -Value '<html> <head> <title>Amazon ECS Sample App</title> <style>body {margin-top: 40px; background-color: #333;} </style> </head><body> <div style=color:white;text-align:center> <h1>Amazon ECS Sample App</h1> <h2>Congratulations!</h2> <p>Your application is now running on a container in Amazon ECS.</p>' -Force ; C:\\ServiceMonitor.exe w3svc" ], "portMappings": [ { "protocol": "tcp", "containerPort": 80, "hostPort": 8080 } ] } ], "taskRoleArn": "arn:aws:iam::111122223333:role/ecs-exec-demo-task-role", "executionRoleArn": "arn:aws:iam::111122223333:role/ecsTaskExecutionRole" }
-
Register the task definition by running the following command:
The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws ecs register-task-definition \ --cli-input-json file://
windows-gmsa-domainless-task-def.json
Step 10: Register a Windows container instance to the cluster
Launch an Amazon EC2 Windows instance and run the ECS container agent to register it as a container instance in the cluster. ECS runs tasks on the container instances that are registered to the cluster that the tasks are started in.
-
To launch an Amazon EC2 Windows instance that is configured for Amazon ECS in the AWS Management Console, see Launching an Amazon ECS Windows container instance. Stop at the step for user data.
-
For gMSA, the user data must set the environment variable
ECS_GMSA_SUPPORTED
before starting the ECS container agent.For ECS Exec, the agent must start with the argument
-EnableTaskIAMRole
.To secure the instance IAM role by preventing tasks from reaching the EC2 IMDS web service to retrieve the role credentials, add the argument
-AwsvpcBlockIMDS
. This only applies to tasks that use theawsvpc
network mode.<powershell> [Environment]::SetEnvironmentVariable("ECS_GMSA_SUPPORTED", $TRUE, "Machine") Import-Module ECSTools Initialize-ECSAgent -Cluster
windows-domainless-gmsa-cluster
-EnableTaskIAMRole -AwsvpcBlockIMDS </powershell> -
Review a summary of your instance configuration in the Summary panel, and when you're ready, choose Launch instance.
Step 11: Verify the container instance
You can verify that there is a container instance in the cluster using the AWS Management Console. However, gMSA needs additional features that are indicated as attributes. These attributes aren't visible in the AWS Management Console, so this tutorial uses the AWS CLI.
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
List the container instances in the cluster. Container instances have an ID that is different from the ID of the EC2 instance.
$
aws ecs list-container-instances
Output:
{ "containerInstanceArns": [ "arn:aws:ecs:
aws-region
:111122223333:container-instance/default
/MyContainerInstanceID
" ] }For example,
526bd5d0ced448a788768334e79010fd
is a valid container instance ID. -
Use the container instance ID from the previous step to get the details for the container instance. Replace
MyContainerInstanceID
with the ID.The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws ecs describe-container-instances \ ----container-instances MyContainerInstanceID
Note that the output is very long.
-
Verify that the
attributes
list has an object with the key calledname
and a valueecs.capability.gmsa-domainless
. The following is an example of the object.Output:
{ "name": "ecs.capability.gmsa-domainless" }
Step 12: Run a Windows task
Run an Amazon ECS task. If there is only 1 container instance in the cluster, you can use
run-task
. If there are many different container instances, it might be easier
to use start-task
and specify the container instance ID to run the task on, than
to add placement constraints to the task definition to control what type of container instance
to run this task on.
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.aws ecs run-task --task-definition windows-gmsa-domainless-task \ --enable-execute-command --cluster windows-domainless-gmsa-cluster
Note the task ID that is returned by the command.
-
Run the following command to verify that the task has started. This command waits and doesn't return the shell prompt until the task starts. Replace
MyTaskID
with the task ID from the previous step.$
aws ecs wait tasks-running --task MyTaskID
Step 13: Verify the container has gMSA credentials
Verify that the container in the task has a Kerberos token. gMSA
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws ecs execute-command \ --task MyTaskID \ --container windows_sample_app \ --interactive \ --command powershell.exe
The output will be a PowerShell prompt.
-
Run the following command in the PowerShell terminal inside the container.
PS C:\>
klist get ExampleAccount$
In the output, note the
Principal
is the one that you created previously.
Step 14: Clean up
When you are finished with this tutorial, you should clean up the associated resources to avoid incurring charges for unused resources.
This step uses the AWS CLI. You can run these commands in
AWS CloudShell in the default shell, which is bash
.
-
Stop the task. Replace
MyTaskID
with the task ID from step 12, Step 12: Run a Windows task.$
aws ecs stop-task --task
MyTaskID
-
Terminate the Amazon EC2 instance. Afterwards, the container instance in the cluster will be deleted automatically after one hour.
You can find and terminate the instance by using the Amazon EC2 console. Or, you can run the following command. To run the command, find the EC2 instance ID in the output of the
aws ecs describe-container-instances
command from step 1, Step 11: Verify the container instance. i-10a64379 is an example of an EC2 instance ID.$
aws ec2 terminate-instances --instance-ids
MyInstanceID
-
Delete the CredSpec file in Amazon S3. Replace
MyBucket
with the name of your Amazon S3 bucket.$
aws s3api delete-object --bucket
MyBucket
--keyecs-domainless-gmsa-credspec/gmsa-cred-spec.json
-
Delete the secret from Secrets Manager. If you used SSM Parameter Store instead, delete the parameter.
The following command uses backslash continuation characters that are used by
sh
and compatible shells. This command isn't compatible with PowerShell. You must modify the command to use it with PowerShell.$
aws secretsmanager delete-secret --secret-id
gmsa-plugin-input
\ --force-delete-without-recovery -
Deregister and delete the task definition. By deregistering the task definition, you mark it as inactive so it can't be used to start new tasks. Then, you can delete the task definition.
-
Deregister the task definition by specifying the version. ECS automatically makes versions of task definitions, that are numbered starting from 1. You refer to the versions in the same format as the labels on container images, such as
:1
.$
aws ecs deregister-task-definition --task-definition
windows-gmsa-domainless-task:1
-
Delete the task definition.
$
aws ecs delete-task-definitions --task-definition
windows-gmsa-domainless-task:1
-
-
(Optional) Delete the ECS cluster, if you created a cluster.
$
aws ecs delete-cluster --cluster
windows-domainless-gmsa-cluster
Debugging Amazon ECS domainless gMSA for Windows containers
- Amazon ECS task status
-
ECS tries to start a task exactly once. Any task that has an issue is stopped, and set to the status
STOPPED
. There are two common types of issues with tasks. First, tasks that couldn't be started. Second, tasks where the application has stopped inside one of the containers. In the AWS Management Console, look at the Stopped reason field of the task for the reason why the task was stopped. In the AWS CLI, describe the task and look at thestoppedReason
. For steps in the AWS Management Console and AWS CLI, see Viewing Amazon ECS stopped task errors. - Windows Events
-
Windows Events for gMSA in containers are logged in the
Microsoft-Windows-Containers-CCG
log file and can be found in the Event Viewer in the section Applications and Services inLogs\Microsoft\Windows\Containers-CCG\Admin
. For more debugging tips, see Troubleshoot gMSAs for Windows containerson the Microsoft Learn website. - ECS agent gMSA plugin
-
Logging for gMSA plugin for the ECS agent on the Windows container instance is in the following directory,
C:/ProgramData/Amazon/gmsa-plugin/
. Look in this log to see if the domainless user credentials were downloaded from the storage location, such as Secrets Manager, and that the credential format was read correctly.