Learn technical details about the SSM Agent

Use the information in this topic to help you implement AWS Systems Manager Agent (SSM Agent) and understand how the agent works.

Topics

• SSM Agent version 3.2.x.x credential behavior
• SSM Agent credentials precedence
• About the local ssm-user account
• SSM Agent and the Instance Metadata Service (IMDS)
• Keeping SSM Agent up-to-date
• Ensuring that the SSM Agent installation directory is not modified, moved, or deleted
• SSM Agent rolling updates by AWS Regions
• SSM Agent communications with AWS managed S3 buckets
• Find AMIs with the SSM Agent preinstalled
• Working with SSM Agent on EC2 instances for Linux
• Working with SSM Agent on EC2 instances for macOS
• Working with SSM Agent on EC2 instances for Windows Server
• Checking SSM Agent status and starting the agent
• Checking the SSM Agent version number
• Viewing SSM Agent logs
• Restricting access to root-level commands through SSM Agent
• Automating updates to SSM Agent
• Subscribing to SSM Agent notifications

SSM Agent version 3.2.x.x credential behavior

SSM Agent stores a set of temporary credentials at /var/lib/amazon/ssm/credentials (for Linux and macOS) or %PROGRAMFILES%\Amazon\SSM\credentials (for Windows Server) when an instance is onboarded using the Default Host Management Configuration in Quick Setup. The temporary credentials have the permissions you specify for the IAM role you chose for Default Host Management Configuration. On Linux, only the root account can access these credentials. On Windows Server, only the SYSTEM account and local Administrators can access these credentials.

SSM Agent credentials precedence

This topic describes important information about how SSM Agent is granted permission to perform actions on your resources.

Note

Support for edge devices differs slightly. You must configure your edge devices to use AWS IoT Greengrass Core software, configure an AWS Identity and Access Management (IAM) service role, and deploy SSM Agent to your devices by using AWS IoT Greengrass. For more information, see Managing edge devices with Systems Manager.

When SSM Agent is installed on an machine, it requires permissions in order to communicate with the Systems Manager service. On Amazon Elastic Compute Cloud (Amazon EC2) instances, these permissions are provided in an instance profile that is attached to the instance. On a non-EC2 machine, SSM Agent normally gets the needed permissions from the shared credentials file, located at /root/.aws/credentials (Linux and macOS) or %USERPROFILE%\.aws\credentials (Windows Server). The needed permissions are added to this file during the hybrid activation process.

In rare cases, however, a machine might end up with permissions added to more than one of the locations where SSM Agent checks for permissions to run its tasks.

For example, say that you have configured an EC2 instance to be managed by Systems Manager. That configuration includes attaching an instance profile. But then you decide to also use that instance for developer or end-user tasks and install the AWS Command Line Interface (AWS CLI) on it. This installation results in additional permissions being added to a credentials file on the instance.

When you run a Systems Manager command on the instance, SSM Agent might try to use credentials different from the ones you expect it to use, such as from a credentials file instead of an instance profile. This is because SSM Agent looks for credentials in the order prescribed for the default credential provider chain.

Note

On Linux and macOS, SSM Agent runs as the root user. Therefore, the environment variables and credentials file that SSM Agent looks for in this process are those of the root user only (/root/.aws/credentials). SSM Agent doesn't look at the environment variables or credentials file of any other users on the instance during the search for credentials.

The default provider chain looks for credentials in the following order:

1. Environment variables, if configured (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY).

2. Shared credentials file ($HOME/.aws/credentials for Linux and macOS or %USERPROFILE%\.aws\credentials for Windows Server) with permissions provided by, for example, a hybrid activation or an AWS CLI installation.

3. An AWS Identity and Access Management (IAM) role for tasks if an application is present that uses an Amazon Elastic Container Service (Amazon ECS) task definition or RunTask API operation.

4. An instance profile attached to an Amazon EC2 instance.

5. The IAM role chosen for Default Host Management Configuration.

For related information, see the following topics:

• Instance profiles for EC2 instances – Configure instance permissions required for Systems Manager

• Hybrid activations – Create a hybrid activation to register nodes with Systems Manager

• AWS CLI credentials – Configuration and credential file settings in the AWS Command Line Interface User Guide

• Default credential provider chain – Specifying Credentials in the AWS SDK for Go Developer Guide

  Note

  This topic in the AWS SDK for Go Developer Guide describes the default provider chain in terms of the SDK for Go; however, the same principles apply to evaluating credentials for SSM Agent.

About the local ssm-user account

Starting with version 2.3.50.0 of SSM Agent, the agent creates a local user account called ssm-user and adds it to the /etc/sudoers.d directory (Linux and macOS) or to the Administrators group (Windows Server). On agent versions before 2.3.612.0, the account is created the first time SSM Agent starts or restarts after installation. On version 2.3.612.0 and later, the ssm-user account is created the first time a session is started on an instance. This ssm-user is the default OS user when a session starts in Session Manager, a capability of AWS Systems Manager. You can change the permissions by moving ssm-user to a less-privileged group or by changing the sudoers file. The ssm-user account isn't removed from the system when SSM Agent is uninstalled.

On Windows Server, SSM Agent handles setting a new password for the ssm-user account when each session starts. No passwords are set for ssm-user on Linux managed instances.

Starting with SSM Agent version 2.3.612.0, the ssm-user account isn't created automatically on Windows Server machines that are being used as domain controllers. To use Session Manager on a Windows Server domain controller, create the ssm-user account manually if it isn't already present, and assign Domain Administrator permissions to the user.

Important

In order for the ssm-user account to be created, the instance profile attached to the instance must provide the necessary permissions. For information, see Step 2: Verify or add instance permissions for Session Manager.

SSM Agent and the Instance Metadata Service (IMDS)

Systems Manager relies on EC2 instance metadata to function correctly. Systems Manager can access instance metadata using either version 1 or version 2 of the Instance Metadata Service (IMDSv1 and IMDSv2). Your instance must be able to access IPv4 address of the instance metadata service: 169.254.169.254. For more information, see Instance metadata and user data in the Amazon EC2 User Guide.

Keeping SSM Agent up-to-date

An updated version of SSM Agent is released whenever new capabilities are added to Systems Manager or updates are made to existing capabilities. Failing to use the latest version of the agent can prevent your managed node from using various Systems Manager capabilities and features. For that reason, we recommend that you automate the process of keeping SSM Agent up to date on your machines. For information, see Automating updates to SSM Agent. Subscribe to the SSM Agent Release Notes page on GitHub to get notifications about SSM Agent updates.

Note

An updated version of SSM Agent is released whenever new capabilities are added to Systems Manager or updates are made to existing capabilities. Failing to use the latest version of the agent can prevent your managed node from using various Systems Manager capabilities and features. For that reason, we recommend that you automate the process of keeping SSM Agent up to date on your machines. For information, see Automating updates to SSM Agent. Subscribe to the SSM Agent Release Notes page on GitHub to get notifications about SSM Agent updates.

Amazon Machine Images (AMIs) that include SSM Agent by default can take up to two weeks to be updated with the newest version of SSM Agent. We recommend that you configure even more frequent automated updates to SSM Agent.

Ensuring that the SSM Agent installation directory is not modified, moved, or deleted

SSM Agent is installed at /var/lib/amazon/ssm/ (Linux and macOS) and %PROGRAMFILES%\Amazon\SSM\ (Windows Server). These installation directories contain critical files and folders used by SSM Agent, such as a credentials file, resources for inter-process communication (IPC), and orchestration folders. Nothing within the installation directory should be modified, moved, or deleted. Otherwise, SSM Agent might cease to function properly.

SSM Agent rolling updates by AWS Regions

After an SSM Agent update is made available in its GitHub repository, it can take up to two weeks until the updated version is rolled out to all AWS Regions at different times. For this reason, you might receive the "Unsupported on current platform" or "updating amazon-ssm-agent to an older version, please turn on allow downgrade to proceed" error when trying to deploy a new version of SSM Agent in a Region.

To determine the version of SSM Agent available to you, you can run a curl command.

To view the version of the agent available in the global download bucket, run the following command.

curl https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/VERSION

To view the version of the agent available in a specific Region, run the following command, substituting region with the Region you're working in, such as us-east-2 for the US East (Ohio) Region.

curl https://s3.region.amazonaws.com/amazon-ssm-region/latest/VERSION

You can also open the VERSION file directly in your browser without a curl command.

SSM Agent communications with AWS managed S3 buckets

In the course of performing various Systems Manager operations, AWS Systems Manager Agent (SSM Agent) accesses a number of Amazon Simple Storage Service (Amazon S3) buckets. These S3 buckets are publicly accessible, and by default, SSM Agent connects to them using HTTP calls.

However, if you're using a virtual private cloud (VPC) endpoint in your Systems Manager operations, you must provide explicit permission in an Amazon Elastic Compute Cloud (Amazon EC2) instance profile for Systems Manager, or in a service role for non-EC2 machines in a hybrid and multicloud environment. Otherwise, your resources can't access these public buckets.

To grant your managed nodes access to these buckets when you are using a VPC endpoint, you create a custom Amazon S3 permissions policy, and then attach it to your instance profile (for EC2 instances) or your service role (for non-EC2 managed nodes).

For information about using a virtual private cloud (VPC) endpoint in your Systems Manager operations, see Improve the security of EC2 instances by using VPC endpoints for Systems Manager.

Note

These permissions only provide access to the AWS managed buckets required by SSM Agent. They don't provide the permissions that are necessary for other Amazon S3 operations. They also don't provide permission to your own S3 buckets.

For more information, see the following topics:

• Configure instance permissions required for Systems Manager

• Create the IAM service role required for Systems Manager in hybrid and multicloud environments

Required bucket permissions

The following table describes each of the S3 buckets that SSM Agent might need to access for Systems Manager operations.

Note

region represents the identifier for an AWS Region supported by AWS Systems Manager, such as us-east-2 for the US East (Ohio) Region. For a list of supported region values, see the Region column in Systems Manager service endpoints in the Amazon Web Services General Reference.

Amazon S3 permissions required by SSM Agent

S3 bucket ARN	Description
arn:aws:s3:::aws-windows-downloads-region/*	Required for some SSM documents that support only Windows Server operating systems, plus some for cross-platform support, such as AWSEC2-ConfigureSTIG.
arn:aws:s3:::amazon-ssm-region/*	Required for updating SSM Agent installations. These buckets contain the SSM Agent installation packages, and the installation manifests that are referenced by the AWS-UpdateSSMAgent document and plugin. If these permissions aren't provided, the SSM Agent makes an HTTP call to download the update.
arn:aws:s3:::amazon-ssm-packages-region/*	Required for using versions of SSM Agent prior to 2.2.45.0 to run the SSM document AWS-ConfigureAWSPackage.
arn:aws:s3:::region-birdwatcher-prod/*	Provides access to the distribution service used by version 2.2.45.0 and later of SSM Agent. This service is used to run the document AWS-ConfigureAWSPackage. This permission is needed for all AWS Regions except the Africa (Cape Town) Region (af-south-1) and the Europe (Milan) Region (eu-south-1).
arn:aws:s3:::aws-ssm-distributor-file-region/*	Provides access to the distribution service used by version 2.2.45.0 and later of SSM Agent. This service is used to run the SSM document AWS-ConfigureAWSPackage. This permission is needed only for the Africa (Cape Town) Region (af-south-1) and the Europe (Milan) Region (eu-south-1).
arn:aws:s3:::aws-ssm-document-attachments-region/*	Provides access to the S3 bucket containing the packages for Distributor, a capability of AWS Systems Manager, that are owned by AWS.
arn:aws:s3:::patch-baseline-snapshot-region/*	Provides access to the S3 bucket containing patch baseline snapshots. This is required if you use any of the following SSM documents: AWS-RunPatchBaseline AWS-RunPatchBaselineAssociation AWS-RunPatchBaselineWithHooks AWS-ApplyPatchBaseline (a legacy SSM document) Note In the Middle East (Bahrain) Region (me-south-1) only, this S3 bucket uses a different naming convention. For this AWS Region only, use the following bucket instead. patch-baseline-snapshot-me-south-1-uduvl7q8 In the Africa (Cape Town) Region (af-south-1) only, this S3 bucket uses a different naming convention. For this AWS Region only, use the following bucket instead. patch-baseline-snapshot-af-south-1-tbxdb5b9
For Linux and Windows Server managed nodes: arn:aws:s3:::aws-ssm-region/* For Amazon EC2 instances for macOS: arn:aws:s3:::aws-patchmanager-macos-region/*	Provides access to the S3 bucket containing modules required for use with certain Systems Manager documents (SSM documents). For example: arn:aws:s3:::aws-ssm-us-east-2/* aws-patchmanager-macos-us-east-2/* Exceptions The S3 bucket names in a few AWS Regions use an extended naming convention, as shown by their ARNs. For these Regions, use the following ARNs instead: Middle East (Bahrain) Region (me-south-1)): aws-patch-manager-me-south-1-a53fc9dce Africa (Cape Town) Region (af-south-1): aws-patch-manager-af-south-1-bdd5f65a9 Europe (Milan) Region (eu-south-1): aws-patch-manager-eu-south-1-c52f3f594 Asia Pacific (Osaka) Region (ap-northeast-3): aws-patch-manager-ap-northeast-3-67373598a SSM documents The following are some commonly used SSM documents stored in these buckets. In arn:aws:s3:::aws-ssm-region/: AWS-RunPatchBaseline AWS-RunPatchBaselineAssociation AWS-RunPatchBaselineWithHooks AWS-InstanceRebootWithHooks AWS-ConfigureWindowsUpdate AWS-FindWindowsUpdates AWS-PatchAsgInstance AWS-PatchInstanceWithRollback AWS-UpdateSSMAgent AWS-UpdateEC2Config In arn:aws:s3:::aws-patchmanager-macos-region/: AWS-RunPatchBaseline AWS-RunPatchBaselineAssociation AWS-RunPatchBaselineWithHooks AWS-InstanceRebootWithHooks AWS-PatchAsgInstance AWS-PatchInstanceWithRollback

Example

The following example illustrates how to provide access to the S3 buckets required for Systems Manager operations in the US East (Ohio) Region (us-east-2). In most cases, you need to provide these permissions explicitly in an instance profile or service role only when using a VPC endpoint.

Important

We recommend that you avoid using wildcard characters (*) in place of specific Regions in this policy. For example, use arn:aws:s3:::aws-ssm-us-east-2/* and don't use arn:aws:s3:::aws-ssm-*/*. Using wildcards could provide access to S3 buckets that you don’t intend to grant access to. If you want to use the instance profile for more than one Region, we recommend repeating the first Statement block for each Region.

{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": "s3:GetObject",
                "Resource": [
                    "arn:aws:s3:::aws-windows-downloads-us-east-2/*",
                    "arn:aws:s3:::amazon-ssm-us-east-2/*",
                    "arn:aws:s3:::amazon-ssm-packages-us-east-2/*",
                    "arn:aws:s3:::us-east-2-birdwatcher-prod/*",
                    "arn:aws:s3:::aws-ssm-document-attachments-us-east-2/*",
                    "arn:aws:s3:::patch-baseline-snapshot-us-east-2/*",
                    "arn:aws:s3:::aws-ssm-us-east-2/*",
                    "arn:aws:s3:::aws-patchmanager-macos-us-east-2/*"
                ]
            }
        ]
    }

Validating hybrid-activated machines using a hardware fingerprint

When non-EC2 machines in a hybrid and multicloud environment, SSM Agent gathers a number of system attributes (referred to as the hardware hash) and uses these attributes to compute a fingerprint. The fingerprint is an opaque string that the agent passes to certain Systems Manager APIs. This unique fingerprint associates the caller with a particular hybrid-activated managed node. The agent stores the fingerprint and hardware hash on the local disk in a location called the Vault.

The agent computes the hardware hash and fingerprint when the machine is registered for use with Systems Manager. Then, the fingerprint is passed back to the Systems Manager service when the agent sends a RegisterManagedInstance command.

Later, when sending a RequestManagedInstanceRoleToken command, the agent checks the fingerprint and hardware hash in the Vault to make sure that the current machine attributes match with the stored hardware hash. If the current machine attributes do match the hardware hash stored in the Vault, the agent passes in the fingerprint from the Vault to RegisterManagedInstance, resulting in a successful call.

If the current machine attributes don't match the stored hardware hash, SSM Agent computes a new fingerprint, stores the new hardware hash and fingerprint in the Vault, and passes the new fingerprint to RequestManagedInstanceRoleToken. This causes RequestManagedInstanceRoleToken to fail, and the agent won't be able to obtain a role token to connect to the Systems Manager service.

This failure is by design and is used as a verification step to prevent multiple managed nodes from communicating with the Systems Manager service as the same managed node.

When comparing the current machine attributes to the hardware hash stored in the Vault, the agent uses the following logic to determine whether the old and new hashes match:

• If the SID (system/machine ID) is different, then no match.

• Otherwise, if the IP address is the same, then match.

• Otherwise, the percentage of machine attributes that match is computed and compared with the user-configured similarity threshold to determine whether there is a match.

The similarity threshold is stored in the Vault, as part of the hardware hash.

The similarity threshold can be set after an instance is registered using a command like the following.

On Linux machines:

sudo amazon-ssm-agent -fingerprint -similarityThreshold 1

On Windows Server machines using PowerShell:

cd "C:\Program Files\Amazon\SSM\" `
    .\amazon-ssm-agent.exe -fingerprint -similarityThreshold 1

Important

If one of the components used to calculate the fingerprint changes, this can cause the agent to hibernate. To help avoid this hibernation, set the similarity threshold to a low value, such as 1.

SSM Agent on GitHub

The source code for SSM Agent is available on GitHub so that you can adapt the agent to meet your needs. We encourage you to submit pull requests for changes that you would like to have included. However, Amazon Web Services doesn't provide support for running modified copies of this software.