Software installers to build custom AMIs for AWS PCS

AWS provides a downloadable file that can install the AWS PCS software on an instance. AWS also provides software that can download, compile, and install relevant versions of Slurm and its dependencies. You can use these instructions to build custom AMIs for use with AWS PCS or you can use your own methods.

Contents

• AWS PCS software installer
• Slurm installer
• Supported operating systems
• Supported instance types
• Supported Slurm versions
• Verify installers using a checksum

AWS PCS software installer

The AWS PCS software installer configures an instance to work with AWS PCS during the instance bootstrap process. You must use AWS-provided installers to install the AWS PCS software on your custom AMI.

Slurm installer

The Slurm installer downloads, compiles, and installs relevant versions of Slurm and its dependencies. You can use the Slurm installer to build custom AMIs for AWS PCS. You can also use your own mechanisms if they are consistent with the software configuration that the Slurm installer provides.

The AWS-provided software installs the following:

• Slurm at the requested major and maintenance version (currently version 23.11.10) - License GPL 2

  • Slurm is built with --sysconfdir set to /etc/slurm

  • Slurm is built with the option --enable-pam and --without-munge

  • Slurm is built with the option --sharedstatedir=/run/slurm/

  • Slurm is built with PMIX and JWT support

  • Slurm is installed at /opt/aws/pcs/schedulers/slurm-23.11

• OpenPMIX (version 4.2.6) – License

  • OpenPMIX is installed as a subdirectory of /opt/aws/pcs/scheduler/

• libjwt (version 1.17.0) – License MPL-2.0

  • libjwt is installed as a subdirectory of /opt/aws/pcs/scheduler/

The AWS-provided software changes the system configuration as follows:

• The Slurm systemd file created by the build is copied to /etc/systemd/system/ with file name slurmd-23.11.service.

• If they don't exist, a Slurm user and group (slurm:slurm) are created with UID/GID of 401.

• On Amazon Linux 2 and Rocky Linux 9 the installation adds the EPEL repository to install the required software to build Slurm or its dependencies.

• On RHEL9 the installation will enable codeready-builder-for-rhel-9-rhui-rpms and epel-release-latest-9 from fedoraproject to install the required software to build Slurm or its dependencies.

Supported operating systems

The AWS PCS software and Slurm installers suppport the following operating systems:

• Amazon Linux 2

• RedHat Enterprise Linux 9

• Rocky Linux 9

• Ubuntu 22.04

Note

AWS Deep Learning AMIs (DLAMI) versions based on Amazon Linux 2 and Ubuntu 22.04 should be compatible with the AWS PCS software and Slurm installers. For more information, see Choosing Your DLAMI in the AWS Deep Learning AMIs Developer Guide.

Supported instance types

AWS PCS software and Slurm installers support any x86_64 or arm64 instance type than can run one of the supported operating systems.

Supported Slurm versions

The following major versions of Slurm are supported:

• Slurm 23.11

Verify installers using a checksum

You can use SHA256 checksums to verify the installer tarball (.tar.gz) files. We recommend that you do this to verify the identity of the software publisher and to check that the application has not been altered or corrupted since it was published.

To verify a tarball

Use the sha256sum utility for the SHA256 checksum and specify the tarball filename. You must run the command from the directory where you saved the tarball file.

• SHA256

  $ sha256sum tarball_filename.tar.gz

The command should return a checksum value in the following format.

checksum_value tarball_filename.tar.gz

Compare the checksum value returned by the command with the checksum value provided in the following table. If the checksums match, then it's safe to run the installation script.

Important

If the checksums don't match, don't run the installation script. Contact AWS Support.

For example, the following command generates the SHA256 checksum for the Slurm 23.11.10 tarball.

$ sha256sum aws-pcs-slurm-23.11-installer-23.11.10-1.tar.gz

Example output:

1de7d919c8632fe8e2806611bed4fde1005a4fadc795412456e935c7bba2a9b8 aws-pcs-slurm-23.11-installer-23.11.10-1.tar.gz

The following table lists the checksums for recent versions of the installers. Replace us-east-1 with the AWS Region where you use AWS PCS.

Installer	Download URL	SHA256 checksum
AWS PCS agent 1.0.1-1	https://aws-pcs-repo-us-east-1.s3.amazonaws.com/aws-pcs-agent/aws-pcs-agent-v1.0.1-1.tar.gz	04e22264019837e3f42d8346daf5886eaacecd21571742eb505ea8911786bcb2
AWS PCS agent 1.0.0-1	https://aws-pcs-repo-us-east-1.s3.amazonaws.com/aws-pcs-agent/aws-pcs-agent-v1.0.0-1.tar.gz	d2d3d68d00c685435c38af471d7e2492dde5ce9eb222d7b6ef0042144b134ce0
Slurm 23.11.10-1	https://aws-pcs-repo-us-east-1.s3.amazonaws.com/aws-pcs-slurm/aws-pcs-slurm-23.11-installer-23.11.10-1.tar.gz	27e8faa9980e92cdfd8cfdc71f937777f0934552ce61e33dac4ecf5a20321e44
Slurm 23.11.9-1	https://aws-pcs-repo-us-east-1.s3.amazonaws.com/aws-pcs-slurm/aws-pcs-slurm-23.11-installer-23.11.9-1.tar.gz	1de7d919c8632fe8e2806611bed4fde1005a4fadc795412456e935c7bba2a9b8