# Release notes and document history
<a name="document_history"></a>

The following tables describe the major updates and new features for the *AWS ParallelCluster User Guide*. We also update the documentation frequently to address the feedback that you send us.


**AWS ParallelCluster**  

| Change | Description | Date | 
| --- | --- | --- | 
| AWS ParallelCluster version 3.16.0 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Enhancements:+ Add the `pcluster-diag` diagnostics tool to AWS ParallelCluster AMIs for running on-demand diagnostic checks on cluster nodes.<br />+ Improve resilience of cluster creation by retrying EBS volume attachment on transient IMDS connectivity failures.<br />+ Further reduce transient build-image failures on RHEL and Rocky caused by out-of-sync repo mirrors by resetting metadata upon retry.<br />+ Improve resilience of cluster updates on login nodes by reusing the head-node-driven orchestration already in place on compute nodes, removing the dependency on cfn-hup and cfn-init.<br />+ Move all AWS ParallelCluster-managed bootstrap files off `/tmp` into a dedicated `/opt/parallelcluster/tmp` directory, so that image builds, cluster creations, and updates work on custom AMIs that mount `/tmp` with noexec. Image builds work on the custom AMIs only if GDRcopy installation is skipped.<br />+ Disable `fwupd-refresh.timer` to improve performance for tightly coupled workloads at scale.<br />Changes:+ The validator `ClusterNameValidator` now limits cluster names to 40 characters when using `ExternalSlurmdbd`, consistent with the existing limit for `Database`. This prevents runtime failures caused by MySQL's table name length limit.<br />+ The validator `MultiNetworkInterfacesInstancesValidator` now also covers single-network-card instances with EFA enabled, which are launched with multiple network interfaces and therefore cannot be auto-assigned a public IP.<br />+ The CLI now requires the additional permission `tag:GetResources`, which it uses to resolve login nodes load balancer ARNs by tags (see the related bugfix).<br />+ Add support for Python 3.13 in the pcluster CLI.<br />+ Enforce NFSv4-only on the AWS ParallelCluster-managed NFS server (head node). The NFSv3 client stack (rpcbind, rpc-statd, lockd) is unchanged, so cluster nodes can still mount external NFSv3 servers.<br />+ Change the default NFS lock manager (lockd) port from 32768 to 4045, to avoid collisions with the Linux ephemeral port range (32768–60999), which can cause sporadic mount failures. This only affects nodes that mount an external NFSv3 server; all AWS ParallelCluster-managed storage is mounted over NFSv4 and is unaffected. If you restrict NFS ports in a firewall, you must open TCP/UDP 4045 or NFSv3 mounts will fail.<br />+ Install the NVIDIA driver, CUDA toolkit, Fabric Manager, NVLSM, and IMEX from the distribution package manager using NVIDIA local repo packages instead of the run file installers.<br />+ On RHEL-family OSes, install the Xorg driver for DCV GPU acceleration and disable Wayland so that GDM always starts Xorg. This is now required after switching NVIDIA driver installation to local repo packages; previously it was needed only on Ubuntu.<br />+ Upgrade NVIDIA driver, Fabric Manager, and IMEX to version 595.71.05 (from 580.126.20).<br />+ Upgrade CUDA Toolkit to version 13.2.2 (from 13.0.2).<br />+ Upgrade DCGM to version 4.6.0 (from 4.5.1).<br />+ In GPU Health Check, skip DCGM diagnostics when NVIDIA MIG is enabled because dcgmi diag does not support MIG.<br />+ Upgrade Slurm to version 25.11.6 (from 25.11.4).<br />+ Upgrade EFA installer to 1.49.0 (from 1.47.0).  Efa-driver: efa-3.1.0 Efa-config: efa-config-1.18-1 Efa-profile: efa-profile-1.7-1 Libfabric-aws: libfabric-aws-2.4.0-1 Rdma-core: rdma-core-63.0-1 Open MPI: openmpi40-aws-4.1.7-3 and openmpi50-aws-5.0.9-11  <br />+ Upgrade mysql-community-client to version 8.4.10 (from 8.4.8).<br />+ Install the aws-parallelcluster-node package from Amazon S3 in all regions instead of PyPI, to support air-gapped and proxied environments.<br />+ Install amazon-efs-utils from the official Amazon EFS endpoint instead of building from source. Users must now allowlist the CloudFront domain `amazon-efs-utils.aws.com` in their proxy/egress configuration when using `build-image` with isolated subnets.<br />+ Upgrade Cinc Client to version 19.3.14 (from 18.8.54).<br />+ Upgrade GDRCopy to version 2.6 (from 2.5.2).<br />+ Upgrade PMIx to version 5.0.11 (from 5.0.10).<br />+ Upgrade Enroot to version 4.2.1 (from 3.4.1).<br />+ Upgrade Pyxis to version 0.24.0 (from 0.20.0).<br />+ Upgrade stunnel to version 5.78 (from 5.67).<br />+ Upgrade Python to version 3.14.6 (from 3.14.2).<br />+ Upgrade Intel MPI to version 2021.18.0.749 (from 2021.17.2.94).<br />+ Upgrade Arm Performance Libraries (ArmPL) to version 26.01.1 (from 24.10).<br />+ Upgrade aws-cfn-bootstrap to version 2.0-40 (from 2.0-38).<br />+ Upgrade third-party cookbook dependencies:  line-5.0.0 (from line-4.5.21) nfs-5.1.6 (from nfs-5.1.5) openssh-2.11.17 (from openssh-2.11.14) yum-8.0.0 (from yum-7.4.20) yum-epel-5.0.9 (from yum-epel-5.0.8)  <br />+ Raise the maximum allowed value of the FSx for Lustre `AutomaticBackupRetentionDays` parameter from 35 to 90, matching the Amazon FSx service limit.<br />Bug Fixes:+ Fix cluster creation failure when login nodes are used and CloudWatch logging is disabled.<br />+ Fix sporadic Amazon S3 bucket creation failure (for buckets named `parallelcluster-*-v1-do-not-delete`) when multiple create-cluster commands run simultaneously in the same region.<br />+ Fix cluster creation failure caused by Slurm accounting bootstrap failing when ClusterName is overridden via custom Slurm settings or when the cluster name contains uppercase letters.<br />+ Remove the deprecated `AccountingStorageUser` parameter from the Slurm configuration, which was causing harmless error messages.<br />+ Fix DCV configuration by letting the DCV server decide the display encoders for the instance type.<br />+ Fix DCV prerequisite installation potentially blocking on interactive prompts by exporting `DEBIAN_FRONTEND=noninteractive` to child processes.<br />+ Fix Xdcv segfault caused by DCV attempting GL initialization when GPU acceleration is not supported.<br />+ Fix slurmrestd failing to start on AL2023 because the http-parser library was not discoverable by the dynamic linker.<br />+ Fix compute node bootstrap hanging without a clear error when the compute subnet cannot reach DynamoDB.<br />+ Fix clustermgtd failing to detect compute node bootstrap timeouts, which prevented the cluster from entering protected mode.<br />+ Fix race condition in load balancer lookup by using `tag:GetResources` to resolve load balancer ARNs by tags, which previously caused cluster update failures on clusters with login nodes.<br />+ Fail `pcluster build-image` early when the downloaded cookbook version does not match the AWS ParallelCluster CLI version.<br />+ Fix login nodes not mounting `/opt/parallelcluster/shared` when Amazon EFS is used as the internal shared storage type.<br />+ Fix SELinux not being disabled on RHEL-family OSes (kernels >= 6.4) due to a deprecated mechanism being silently ignored by newer kernels.<br />+ Fix `build-image` failure by skipping installation of `fabric-manager`, `gdrcopy`, and `dcgm` if the parent image already has a version installed.<br />+ Fix an issue where compute nodes are replaced when launching a large number of nodes due to eventual consistency.<br />+ Fix a race condition where a fleet start was not enough to exit protected mode. This race condition is triggered when a fleet stop is issued on a fleet transitioning into protected mode while some of its nodes are powering up.<br />+ Fix an issue where static nodes in a maintenance reservation enter a terminate/relaunch loop when being replaced.<br />+ Fix cluster creation and update failures caused by IAM policy tag conditions that required the `parallelcluster:cluster-name` tag to be the only tag on a request rather than merely present. This affected clusters with login nodes as well as stack creation and updates when AWS or CloudFormation attached additional tags to the request.<br />+ Prevent a path traversal when downloading exported cluster and image logs, so a CloudWatch log stream name containing `../` segments can no longer cause files to be written outside the export directory.<br />+ Return a clear error instead of an unexpected fatal exception when the configured AWS profile does not exist.<br />+ Cache getPolicy API calls to reduce validation time of clusters that have additional IAM policies.<br />Deprecations:+ Amazon Linux 2 is no longer supported.<br />+ AWS Batch as a scheduler is no longer supported. | August 20, 2026 | 
| AWS ParallelCluster version 3.15.1 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Changes:+ Patch official AWS ParallelCluster AMIs to address CVE-2026-31431.<br />+ Disable `algif_aead` kernel module on Ubuntu to address CVE-2026-31431.<br />+ Upgrade NVIDIA driver to version 580.126.20 (from 580.105.08) for all OSs except Amazon Linux 2, addressing CVE-2025-33219.<br />+ Upgrade NVIDIA Fabric manager to 580.126.20 (from 580.105.08) for all OSs except Amazon Linux 2.<br />+ Upgrade NVIDIA IMEX to 580.126.20 (from 580.105.08) for all OSs except Amazon Linux 2. | May 28, 2026 | 
| AWS ParallelCluster version 3.15.0 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Enhancements:+ Add support for p6-b300 instances for all OSs except AL2.<br />+ Replace cfn-hup in compute nodes with systemd timer to support in place updates in order to improve performance for tightly coupled workloads at scale.<br />+ Disable `dnf-makecache.timer` to improve performance for tightly coupled workloads on RHEL/Rocky at scale.<br />+ Support updates of `Tags` during cluster-updates.<br />+ Add `LaunchTemplateOverrides` to cluster config to allow network interfaces to be customized by overriding the launch template of a compute resource.<br />+ Add alarm on missing clustermgtd heartbeat.<br />Changes:+ When EFA is enabled, AWS ParallelCluster now configures network interfaces as `interface` and `efa-only` instead of the combined `efa` type. This reduces IP address consumption from one per network card to one per instance.<br />+ Upgrade Slurm to version 25.11.4 (from 24.11.7).<br />+ Upgrade Pmix to 5.0.10 (from 5.0.6).<br />+ Upgrade EFA installer to 1.47.0 (from 1.44.0).<br />+ Upgrade NVIDIA driver to version 580.105.08 (from 570.172.08) for all OSs except Amazon Linux 2.<br />+ Upgrade GDRCopy to version 2.5.2 (from 2.4.4).<br />+ Upgrade DCV to version 2025.0-20103 (from 2024.0-19030).<br />+ Upgrade CUDA Toolkit to version 13.0.2 (from 12.8.1) for all OSs except Amazon Linux 2.<br />+ Upgrade Python to 3.14.2 (from 3.12.11) for all OSs except Amazon Linux 2.<br />+ Upgrade Cinc Client to version 18.8.54 (from 18.7.10).<br />+ Upgrade Intel MPI Library to 2021.17.2 (from 2021.16.0).<br />+ Upgrade DCGM to version 4.5.1 (from 4.4.1) for all OSs except Amazon Linux 2.<br />+ Reduce transient build-image failures in RHEL and Rocky caused by out-of-sync repo mirrors by resetting metadata upon retry.<br />+ Always start clustermgtd on cluster update and compute fleet status update failure, regardless the failure condition.<br />+ Improve resiliency of the cluster update rollback workflow.<br />+ Upgrade NVIDIA Fabric manager to 580.105.08 for all OSs except Amazon Linux 2.<br />+ Upgrade aws-cfn-bootstrap to version 2.0-38 (from 2.0-33).<br />+ Upgrade mysql-community-client to version 8.4.8 (from 8.0.39) for all OSs except Amazon Linux 2.<br />+ Upgrade amazon-efs-utils to version 2.4.0 (from v2.1.0) for Amazon Linux AMI's.<br />+ Upgrade jmespath to \~=1.0 (from \~=0.10).<br />+ Upgrade tabulate to <=0.9.0 (from <=0.8.10).<br />+ Add a validator to warn in case in-place updates have been disabled (via DevSettings) on compute and login nodes.<br />Bug Fixes:+ Fix LoginNodes NLB not being publicly accessible when using public subnet with implicit main route table association.<br />+ Fix a failure when creating a cluster with GPU instances and with DCV enabled but without internet access.<br />+ Fix an issue where cluster creation would intermittently fail due to eventual consistency when the head/compute/login nodes share the same security group.<br />+ Fix build-image failure during ubuntu-desktop installation on a Ubuntu parent image with outdated OS packages.<br />+ Fix validation of `HeadNode/LocalStorage`. This configuration parameter does not support updates.<br />+ Fix validator `PlacementGroupCapacityReservationValidator` to accept capacity reservations with cross-account placement group.<br />+ Fix the CloudWatch agent configuration to ensure proper parsing of timestamps across all log files.<br />+ Fix logging configuration to capture all Slurm health check events (updating log level from WARNING to INFO to prevent missing log entries).<br />+ Improve cluster update resiliency by ensuring the update does not fail on nodes completing the bootstrap during the update.<br />+ Prevent cluster update failure recovery process from running on AWS Batch clusters. This recovery mechanism should only execute on Slurm clusters.<br />Deprecations:+ The `LoginNodes/Pools/Ssh/KeyName` configuration parameter, deprecated since 3.14.0, is no longer supported.<br />+ This is the last AWS ParallelCluster release supporting Amazon Linux 2, as Amazon Linux 2 will reach end of support on June 30, 2026.<br />+ This is the last AWS ParallelCluster release supporting AWS Batch CLI. Starting with v3.16.0, AWS ParallelCluster will no longer support AWS Batch as a scheduler. | March 23, 2026 | 
| AWS ParallelCluster version 3.14.2 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Changes:+ Upgrade munge to version 0.5.18 (from 0.5.16) to address [CVE-2026-25506 ](https://github.com/dun/munge/security/advisories/GHSA-r9cr-jf4v-75gh).<br />+ Upgrade NodeJS version in installer to version 22.22.0 (from 20.18.3). | February 16, 2026 | 
| AWS ParallelCluster version 3.14.1 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Changes:+ Improve cluster update resiliency by ensuring clustermgtd is started after updates complete successfully, or after failed updates where queue reconfiguration succeeded.<br />+ Add chef attribute `cluster/in_place_update_on_fleet_enabled` to disable in-place updates on compute and login nodes and mitigate performance impact at scale.<br />+ Upgrade Slurm to version 24.11.7 (from 24.11.6).<br />+ Upgrade Werkzeug to \~=3.1 (from \~=2.0) to address [CVE-2024-34069](https://nvd.nist.gov/vuln/detail/CVE-2024-34069).<br />+ Upgrade Connexion to \~=2.15.1 (from \~=2.13.0).<br />+ Upgrade Flask to \~=3.1.0 (from >=2.2.5,<2.3).<br />+ Load kernel module `drm_client_lib` before installation of NVIDIA driver, if available on the kernel.<br />+ Reduce dependency footprint by installing the package `sssd-common` rather than `sssd`.<br />+ Upgrade libjwt to version 1.18.4 (from 1.17.0) for all OSes except Amazon Linux 2.<br />+ Upgrade amazon-efs-utils to version 2.4.0 (from v2.3.1).<br />+ Upgrade EFA installer to 1.44.0 (from 1.43.2).  Efa-driver: efa-2.17.3-1 Efa-config: efa-config-1.18-1 Efa-profile: efa-profile-1.7-1 Libfabric-aws: libfabric-aws-2.3.1-1 Rdma-core: rdma-core-59.0-1 Open MPI: openmpi40-aws-4.1.7-2 and openmpi50-aws-5.0.8-11  <br />Bug Fixes:+ Fix an issue where cfn-hup enters an endless loop on the head node after a rollback to a cluster state older than 24 hours, caused by cfn-signal failing to signal an expired wait condition handle.<br />+ Fix race condition where compute nodes could deploy the wrong cluster config version after an update failure.<br />+ Prevent cluster readiness check failures due to instances launched while the check is in progress.<br />+ Fix incorrect timestamp parsing for chef-client.log in CloudWatch Agent configuration.<br />+ Disable snap auto-refresh on Ubuntu during build image to prevent intermittent reboot failures.<br />+ Reduce EFA installation time for Ubuntu by \~20 minutes by only holding kernel packages for the installed kernel.<br />+ Add `GetFunction` and `GetPolicy` permissions to `PClusterBuildImageCleanupRole` to prevent `AccessDenied` errors during build image stack deletion.<br />+ Fix validation error messages when `DevSettings` is null or `DevSettings/InstanceTypesData` is missing required fields. | December 22, 2025 | 
| AWS ParallelCluster version 3.14.0 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Enhancements:+ Include drivers for P6e-GB200 and P6-B200 instances. ParallelCluster sets up a Slurm topology plugin to handle P6e-GB200 UltraServers. See the Limitations section for important additional setup requirements.<br />+ Support `prioritized` and `capacity-optimized-prioritized` Allocation Strategy. This allows users to prioritize subnets for instance placement to optimize costs and performance.<br />+ Add `build-image` support for Amazon Linux 2023 AMIs based on kernel 6.12 (in addition to 6.1).<br />+ Support DCV on Amazon Linux 2023.<br />+ Echo chef-client logs in the instance console when a node fails to bootstrap. This helps with investigating bootstrap failures in cases CloudWatch logs are not available.<br />Limitations:+ P6e-GB200 instances are only tested on Amazon Linux 2023, Ubuntu 22.04 and Ubuntu 24.04.<br />+ Using IMEX on P6e-GB200 requires additional setup. Please refer to the dedicated tutorial in our public documentation.<br />+ P6-B200 instances are only tested on Amazon Linux 2023, RHEL 8 & 9, Rocky 8 & 9, Ubuntu 22.04 and Ubuntu 24.04.<br />+ GPU HealthChecks are not recommended for instances with GPU memory above 320GB (such as p6-b200.48xlarge). Health check duration can exceed 10 minutes, potentially causing job failures and significantly reducing the job throughput.<br />Changes:+ Install nvidia-imex for all OSs except Amazon Linux 2.<br />+ Remove `UnkillableStepTimeout` from slurm.conf and let slurm set this value.<br />+ Upgrade Python runtime used by Lambda functions to Python 3.12 (from 3.9). See the [Lambda Documentation](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html) for important information about Python 3.9 EOL.<br />+ Support encryption of EFS file system used for the head node internal shared storage via a new configuration parameter `HeadNode/SharedStorageEfsSettings/Encrypted`.<br />+ Add validator that warns against using non GPU instances with DCV.<br />+ Upgrade Slurm to version 24.11.6 (from 24.05.8).<br />+ Upgrade EFA installer to 1.43.2 (from 1.41.0).  Efa-driver: efa-2.17.2-1 Efa-config: efa-config-1.18-1 Efa-profile: efa-profile-1.7-1 Libfabric-aws: libfabric-aws-2.1.0-5 Rdma-core: rdma-core-58.0-1 Open MPI: openmpi40-aws-4.1.7-2 and openmpi50-aws-5.0.6-11  <br />+ Upgrade Cinc Client to version 18.4.12 (from 18.2.7).<br />+ Upgrade NVIDIA driver to version 570.172.08 (from 570.86.15) for all OSs except Amazon Linux 2.<br />+ Upgrade CUDA Toolkit to version 12.8.1 (from 12.8.0) for all OSs except Amazon Linux 2.<br />+ Upgrade DCGM to version 4.4.1 (from 3.3.6) for all OSs except Amazon Linux 2.<br />+ Upgrade Python to 3.12.11 (from 3.12.8) for all OSs except Amazon Linux 2.<br />+ Upgrade Python to 3.9.23 (from 3.9.20) for Amazon Linux 2.<br />+ Upgrade Intel MPI Library to 2021.16.0 (from 2021.13.1).<br />+ Upgrade DCV to version 2024.0-19030.<br />+ Upgrade the official ParallelCluster Amazon Linux 2023 AMIs to kernel 6.12 (from 6.1).<br />Bug Fixes:+ Prevent `build-image` stack deletion failures by deploying a global role that automatically deletes the `build-image` stack after images either succeed or fail the build. The role is meant to exist even after the stack has been deleted. See [ https://github.com/aws/aws-parallelcluster/issues/5914](https://github.com/aws/aws-parallelcluster/issues/5914).<br />+ Fix an issue where Security Group validation failed when a rule contained both IPv4 ranges (IpRanges) and security group references (UserIdGroupPairs).<br />+ Fix `build-image` failure on Rocky 9, occurring when the parent image does not ship the latest kernel version on the latest Rocky minor version.<br />+ Fix cluster id mismatch issue which causes cluster update failures when slurm accounting is used.<br />+ Fix a race condition in CloudWatch Agent startup that could cause node bootstrap failures.<br />Deprecations:+ The configuration parameter `LoginNodes/Pools/Ssh/KeyName` has been deprecated, and it will be removed in future releases. The CLI now returns a warning message when it is used in the cluster configuration. See [ https://github.com/aws/aws-parallelcluster/issues/6811](https://github.com/aws/aws-parallelcluster/issues/6811).<br />+ Ubuntu 20.04 is no longer supported. | September 30, 2025 | 
| AWS ParallelCluster version 3.13.2 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Bug fixes:+  Fix a bug which may cause `update-cluster` and `update-compute-fleet` to fail when compute resources reference an expired Capacity Reservation that is no longer accessible via EC2 APIs. <br />+  Fix `build-image` failure on Rocky 9 that occurs when the parent image does not ship the latest kernel version. See [ https://github.com/aws/aws-parallelcluster/issues/6874](https://github.com/aws/aws-parallelcluster/issues/6874).  | June 24, 2025 | 
| AWS ParallelCluster version 3.13.1 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Changes:+  Upgrade Slurm to version 24.05.8. <br />+  Upgrade EFA installer to 1.41.0 (from 1.38.1).   Efa-driver: efa-2.15.0-1   Efa-config: efa-config-1.18-1   Efa-profile: efa-profile-1.7-1   Libfabric-aws: libfabric-aws-2.1.0-1   Rdma-core: rdma-core-57.0-1   Open MPI: openmpi40-aws-4.1.7-2 and openmpi50-aws-5.0.6   <br />+  Upgrade amazon-efs-utils to version 2.3.1 (from v2.1.0) for non-Amazon Linux AMI's. <br />+  Support DCV in us-isob-east-1 and us-iso-east-1. <br />+  Support FSX for Lustre and Ontap in us-isob-east-1 and us-iso-east-1. <br />+  Ensure kernel consistency throughout ParallelCluster image build by pinning at the beginning and unpinning at completion. <br />Bug fixes:+  Fix a bug in the installation of ARM Performance Library that was causing the build image fail in AWS Top Secret and AWS Secret Regions. <br />+  Fix a bug that was preventing the script 'update\_directory\_service\_password.sh' from updating the AD password.  | June 04, 2025 | 
| AWS ParallelCluster version 3.13.0 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Deprecations:+  This is the last ParallelCluster release supporting Ubuntu 20.04 as Ubuntu 20.04 will be in End-Of-Standard-Support in May 2025. <br />Enhancements:+  Add support for Ubuntu 24.04. <br />+  Add support for ap-southeast-7 region. <br />+  Disable unused services cups and wpa\_supplicant from Official ParallelCluster AMIs to improve security. <br />Changes:+  Upgrade Slurm to version 24.05.7. <br />+  Upgrade NVIDIA driver to version 570.86.15 (from 550.127.08) for all OSs except AL2. <br />+  Upgrade CUDA Toolkit to version 12.8.0 (from 12.4.1) for all OSs except AL2. <br />+  Upgrade Python to 3.12.8 for all OSs except AL2 (from 3.9.20). <br />+  On Ubuntu 22.04, install the Nvidia driver with the same compiler version used to compile the kernel. <br />+  Upgrade `aws-cfn-bootstrap` to version 2.0-33. <br />+  Upgrade EFA installer to 1.38.0 (from 1.36.0).   Efa-driver: efa-2.13.0-1   Efa-config: efa-config-1.17-1   Efa-profile: efa-profile-1.7-1   `Libfabric-aws`: `libfabric-aws-1.22.0-1`   Rdma-core: rdma-core-54.0-1   Open MPI: `openmpi40-aws-4.1.7-1` and `openmpi50-aws-5.0.5`   <br />+  Upgrade amazon-efs-utils to version 2.1.0. <br />+  Remove third-party cookbook: apt-7.5.22 and pyenv-4.2.3. <br />+  Upgrade third-party cookbook dependencies:   line-4.5.21 (from line-4.5.13)   nfs-5.1.5 (from nfs-5.1.2)   openssh-2.11.14 (from openssh-2.11.12)   yum-7.4.20 (from yum-7.4.13)   yum-epel-5.0.8 (from yum-epel-5.0.2)   <br />+  Upgrade Pmix to 5.0.6 (from 5.0.3). <br />+  Upgrade ARM PL to version 24.10 (from 23.10). <br />+  Upgrade Python to version 3.12.8 (from 3.9.17) in Lambda layer and installer. <br />+  Upgrade NodeJS to version 20.18.3 (from 18.20.3) in Lambda layer and installer. <br />+  Remove generation of DSA keys for login nodes as DSA, which became unsupported in OpenSSH 9.7\+. <br />+  Set instance ID and instance type information in Slurm upon compute nodes launch. <br />+  Install NVIDIA drivers without the option 'no-cc-version-check', which is now deprecated in the NVIDIA installer. <br />+  Add validator to enforce up to 10- login node pools. <br />+  Update the default root volume size to 45 GB. <br />Bug fixes:+  Remove usage of cfn-init for compute node bootstrapping to reduce node scale up time. <br />+  Fix an issue causing compute node bootstrap failure when a proxy is used. <br />+  On Ubuntu 22.04, install the Nvidia driver with the same compiler version used to compile the kernel to prevent installation failures.- Fix the execution of overriding aws-parallelcluster-node package only on the head node during update. <br />+  Fix an issue where containerized jobs executed through Pyxis/Enroot in a multi-user environment (integrated with Active Directory) would fail. <br />+  Fix usage of authselect causing node bootstrap failures on Rocky 9.5\+ when directory service is used.  | April 01, 2025 | 
| AWS ParallelCluster version 3.12.0 released |  To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`. <br />Enhancements:+   Add new build image configuration section `Build/Installation` to turn on/off NVIDIA software and Lustre client installations. By default, NVIDIA software, although included in official ParallelCluster AMIs, is not installed by `build-image`. By default, Lustre client is installed.  <br />+   The CLI commands `export-cluster-logs` and `export-image-logs` can now by default export the logs to the default ParallelCluster bucket or to the CustomS3Bucket if specified in the config.  <br />+   Extend Amazon DCV support to Ubuntu2204 on ARM instances.  <br />Changes:+  Upgrade NVIDIA driver to version 550.127.08 (from 550.90.07). This addresses a known issue from NVIDIA. For more information, see [Known Issues](https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-550-90-07/index.html#known-issues) in the *NVIDIA Data Center Documentation*. <br />+  Upgrade Amazon DCV to version `2024.0-18131`.   server: `2024.0-18131-1`   xdcv: `2024.0.631-1`   gl: `2024.0.1078-1`   web\_viewer: `2024.0-18131-1`   <br />+  Upgrade EFA installer to `1.36.0`.   Efa-driver: `efa-2.13.0-1`   Efa-config: `efa-config-1.17-1`   Efa-profile: `efa-profile-1.7-1`   Libfabric-aws: `libfabric-aws-1.22.0-1`   Rdma-core: `rdma-core-54.0-1`   Open MPI: `openmpi40-aws-4.1.7-1` and `openmpi50-aws-5.0.5`   <br />+  Auto-restart slurmctld on failure. <br />+  Upgrade mysql-community-client to version 8.0.39. <br />+  Remove support for Python 3.7 and 3.8, which are end of life. <br />Bug fixes:+  Fix an issue where changes in sequence of custom actions scripts were not detected during cluster updates. <br />+  Add missing permissions for the AWS ParallelCluster API to create the service linked roles for Elastic Load Balancing and Auto Scaling, that are required to deploy login nodes. <br />+  Fix an issue in the way we get the region when manage volumes so that it can correctly handle local zone. <br />+  Fix an issue where adding EFS filesystems with `AccessPointIds` during an update would fail. <br />+  Fix an issue where when using PCAPI, cluster update could fail when updating a parameter that is not type `String` (for example, `MaxCount`). <br />+  When mounting an external OpenZFS, it is no longer required to set the outbound rules for ports 111, 2049, 20001, 20002, 20003.  | December 19, 2024 | 
| AWS ParallelCluster version 3.11.1 released | Features:+  Pyxis is now disabled by default, so it must be manually enabled as documented in the product documentation. <br />+  Upgrade Python runtime to version 3.12 in ParallelCluster Lambda Layer. <br />+  Remove version pinning for setuptools to version prior to 70.0.0. <br />+  Upgrade libjwt to version 1.17.0. <br />+ [Full Changelog](https://github.com/aws/aws-parallelcluster/blob/develop/CHANGELOG.md#3111)<br />Bug fixes+  Fix an issue in the way we configure the Pyxis Slurm plugin in ParallelCluster that can lead to job submission failures. <br />+  Fix an issue that was causing failing deployment in configurations with login nodes by add missing permissions required by login nodes in the public template of policies. [https://github.com/aws/aws-parallelcluster/issues/6483](https://github.com/aws/aws-parallelcluster/issues/6483)  | October 21, 2024 | 
| AWS ParallelCluster version 3.11.0 released | Enhancements+ Add support for custom actions on login nodes.<br />+ Allow DCV connection to login nodes.<br />+ Add support for ap-southeast-3 region.<br />+ Add security groups to login node network load balancer.<br />+ Add `AllowedIps` configuration for login nodes.<br />+ Add new configuration `SharedStorage/EfsSettings/AccessPointId` to specify an optional EFS access point for a mount<br />+ Allow up to 10 login node pools.<br />+ Install enroot and pyxis in official pcluster AMIs<br />Changes+ [*BREAKING*] The `loginNodes` field returned by the API `DescribeCluster` and the CLI command `describe-cluster` has been changed from a dictionary to an array to support multiple pools of login nodes. This change breaks backward compatibility, making these operations incompatible with clusters deployed with older versions. <br />+ Upgrade Slurm to 23.11.10 (from 23.11.7).<br />+ Upgrade Pmix to 5.0.3 (from 5.0.2).<br />+ Upgrade EFA installer to `1.34.0`.  Efa-driver: `efa-2.10.0-1` Efa-config: `efa-config-1.17-1` Efa-profile: `efa-profile-1.7-1` Libfabric-aws: `libfabric-aws-1.22.0-1` Rdma-core: `rdma-core-52.0-1` Open MPI: `openmpi40-aws-4.1.6-3` and `openmpi50-aws-5.0.3-11`  <br />+ Upgrade NVIDIA driver to version 550.90.07 (from 535.183.01).<br />+ Upgrade CUDA Toolkit to version 12.4.1 (from 12.2.2).<br />+ Upgrade Python to 3.9.20 (from 3.9.19).<br />+ Upgrade Intel MPI Library to 2021.13.1.769 (from 2021.12.1.8).<br />Bug fixes+ Fix validator `EfaPlacementGroupValidator` so that it does not suggest to configure a Placement Group when Capacity Blocks are used.<br />+ Fix occasional cluster creation failures by ensuring that FSx for Lustre file systems are created after security group rules.<br />+ Fix cluster deletion failure when placement group is enabled.<br />+ Fix issue with login nodes being marked unhealthy when restricting SSH access.<br />+ Fix `retrieve_supported_regions` so that it can get the correct S3 url.<br />+ Fix `describe_images` to use pagination.<br />+ Fix `No route tables found` bug when specifying default VPC subnet to LoginNodes/Networking/SubnetIds. | September 26, 2024 | 
| AWS ParallelCluster version 3.10.1 released | Bug fixes+ Fix image build failure in China regions. | July 8, 2024 | 
| AWS ParallelCluster version 3.10.0 released | Enhancements:+  Add new configuration section `Scheduling/SlurmSettings/ExternalSlurmdbd` to connect the cluster to an external Slurmdbd. <br />+  Add support for build-image in AWS Top Secret and AWS Secret Regions. <br />+  Add support for Amazon Linux 2023. <br />+  Add support for `price-capacity-optimized` as an `AllocationStrategy`. <br />+  Add validator to prevent the use of Placement Groups with Capacity Blocks. <br />Changes:+  CentOS 7 is no longer supported. <br />+  Upgrade Cinc Client to version to 18.4.12 from 18.2.7. <br />+  Upgrade munge to version 0.5.16 (from 0.5.15). <br />+  Upgrade Pmix to 5.0.2 (from 4.2.9). <br />+  Upgrade third-party cookbook dependencies:   apt-7.5.22 (from apt-7.5.14)   openssh-2.11.12 (from openssh-2.11.3)   <br />+  Remove third-party cookbook: selinux-6.1.12. <br />+  Upgrade EFA installer to `1.32.0`.   Efa-driver: `efa-2.8.0-1`   Efa-config: `efa-config-1.16-1`   Efa-profile: `efa-profile-1.7-1`   Libfabric-aws: `libfabric-aws-1.21.0-1`   Rdma-core: `rdma-core-50.0-1`   Open MPI: `openmpi40-aws-4.1.6-3` and `openmpi50-aws-5.0.2-12`   <br />+  Upgrade NVIDIA driver to version 535.183.01 (from 535.154.05). <br />+  Upgrade Python to 3.9.19 (from 3.9.17). <br />+  Upgrade Intel MPI Library to 2021.12.1.8 (from 2021.9.0.43482). <br />Bug fixes:+  Fix Data Repository Associations configuration to make `AutoExportPolicy` and `AutoImportPolicy` optional. <br />+  Fixed an issue during cluster deletion that now completes compute fleet cleanup when instances are either in shutting-down or terminated state. This is to avoid cluster deletion failures for instance types with longer termination cycles. <br />+  Allow cloudwatch dashboard to be enabled and alarms to be disabled in the `Monitoring` section of the cluster config. <br />+  Allow ParallelCluster Custom Resource to suppress validators using `PclusterCluster/SuppressValidators`. <br />+  Removing `/etc/profile.d/pcluster.sh` so that it's not executed at every user login and `cfn_bootstrap_virtualenv` is not added in PATH environment variable. <br />+  Fix ParallelCluster API spec by replacing field `failureReason` with `failures` in `DescribeCluster` response. <br />+  Fix ParallelCluster API spec by adding the CloudFormation stack status that were missing: `IMPORT_*`, `REVIEW_IN_PROGRESS` and `UPDATE_FAILED`. <br />+  Fix an issue that prevented cluster updates from including EFS filesystems with encryption in transit. <br />+  Fix an issue that prevented slurmctld and slurmdbd services from restarting on head node reboot when EFS is used for shared internal data. <br />+  On Ubuntu systems, remove default logrotate configuration for cloud-init log files that clashed with the configuration coming from ParallelCluster. <br />+  Fix image build failure with RHEL 8.10 or newer.  | June 27, 2024 | 
| AWS ParallelCluster version 3.9.3 released | To upgrade, type `sudo pip install --upgrade aws-parallelcluster`<br />Features:+  Added support for `FSx Lustre` as a shared storage type in `us-iso-east-1`. <br />Bug fixes:+  Remove `cloud_dns` from the `SlurmctldParameters` in the Slurm config to avoid Slurm fanout issues. <br />This isn't required, since we set the IP addresses on instance launch.  | June 19, 2024 | 
| AWS ParallelCluster version 3.9.2 released | Features:+  Upgrade Slurm to 23.11.7 (from 23.11.4). <br />+  For more details, see the `CHANGELOG` [`3.9.2`](https://github.com/aws/aws-parallelcluster/blob/develop/CHANGELOG.md#392) on GitHub.  | May 28, 2024 | 
| AWS ParallelCluster version 3.9.1 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`<br />Bug fixes+  Remove recursive deletion of shared storage mountdir when unmounting filesystems as part of update-cluster operation.  | April 11, 2024 | 
| AWS ParallelCluster version 3.9.0 released | To upgrade, enter the following: `sudo pip install --upgrade aws-parallelcluster`<br />Enhancements:+  Add the configuration parameter `DeploymentSettings/DefaultUserHome` to allow users to move the default user's home directory to `/local/home` instead of `/home` (default). <br />+  Permit to update `MinCount`, `MaxCount`, `Queue` and `ComputeResource` configuration parameters without the need to stop the compute fleet. It's now possible to update them by setting `Scheduling/SlurmSettings/QueueUpdateStrategy` to TERMINATE. AWS ParallelCluster will terminate only the nodes removed during a resize of the cluster capacity performed through a cluster update. <br />+  Permit to update the external shared storage of type Efs, FsxLustre, FsxOntap, FsxOpenZfs and FileCache without replacing the compute and login fleet. <br />+  Add support for RHEL9. <br />+  Add support for Rocky Linux 9 as `CustomAmi` created through `build-image` process. No public official AWS ParallelCluster Rocky9 Linux AMI is made available at this time. <br />+  Remove `CommunicationParameters` from the Custom Slurm Settings deny list. <br />+  Add `DeploymentSettings/DisableSudoAccessForDefaultUser` parameter to disable sudo access of default user in supported OSes. <br />+  Changes to FSx for Lustre file systems created by ParallelCluster: Change the Lustre server version to `2.15.` <br />+  Add possibility to choose between Open and Closed Source Nvidia Drivers when building an AMI, through the `['cluster']['nvidia']['kernel_open']` cookbook node attribute. <br />+  \* Add a clustermgtd config option `ec2_instance_missing_max_count` to allow a configurable amount of retries for eventual Amazon EC2 describe instances consistency with run instances.  <br />Changes+  Upgrade Slurm to 23.11.4 (from 23.02.7). <br />+  Upgrade NVIDIA driver to version 535.154.05. <br />+  Add support for Python 3.11, 3.12 in pcluster CLI and aws-parallelcluster-batch-cli. <br />+  Build network interfaces using network card index from `NetworkCardIndex` list of Amazon EC2 DescribeInstances response, instead of looping over `MaximumNetworkCards` range. <br />+  Fail cluster creation when using instance types P3, G3, P2 and G2 because their GPU architecture is not compatible with Open Source Nvidia Drivers (OpenRM) introduced as part of 3.8.0 release. <br />+  Upgrade third-party cookbook dependencies: nfs-5.1.2 (from nfs-5.0.0) <br />+  Upgrade EFA installer to `1.30.0.`   Efa-driver: `efa-2.6.0-1`   Efa-config: `efa-config-1.15-1`   Efa-profile: `efa-profile-1.6-1`   Libfabric-aws: `libfabric-aws-1.19.0`   Rdma-core: `rdma-core-46.0-1`   Open MPI: `openmpi40-aws-4.1.6-2` and `openmpi50-aws-5.0.0-11`   <br />+  Upgrade NICE DCV to version `2023.1-16388.`   server: `2023.1.16388-1`   xdcv: `2023.1.565-1`   gl: `2023.1.1047-1`   web\_viewer: `2023.1.16388-1`   <br />Bug fixes+  Fix issue making job fail when submitted as active directory user from login nodes. The issue was caused by an incomplete configuration of the integration with the external Active Directory on the head node. <br />+  Refactor IAM policies defined in CloudFormation template parallelclutser-policies.yaml to prevent ParallelCluster API deployment failure caused by policies exceeding IAM limits. <br />+  Fix issue making login nodes fail to bootstrap when the head node takes more time than expected in writing keys. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.12.0) package on GitHub. | March 5, 2024 | 
| AWS ParallelCluster version 3.8.0 released | AWS ParallelCluster version 3.8.0 released.<br />Enhancements:+  Add support for Amazon EC2 Capacity Blocks for ML. <br />+  Add support for Rocky Linux 8 as `CustomAmi` created through `build-image` process. No public official AWS ParallelCluster Rocky8 Linux AMI is made available at this time. <br />+  Add `Scheduling/ScalingStrategy` parameter to control the cluster scaling strategy to use when launching Amazon EC2 instances for Slurm compute nodes. Possible values are `all-or-nothing`, `greedy-all-or-nothing`, `best-effort`, with `all-or-nothing` being the default. <br />+  Add `HeadNode/SharedStorageType` parameter to use EFS storage instead of NFS exports from the head node root volume for intra-cluster shared file system resources: ParallelCluster, Intel, Slurm, and `/home` data. This enhancement reduces the load on the head node networking. <br />+  Allow for mounting `/home` as an EFS or FSx external shared storage via the `SharedStorage` section of the config file. <br />+  Add new parameter `SlurmSettings/MungeKeySecretArn` to permit to use an external user-defined MUNGE key from AWS Secrets Manager. <br />+  Add `Monitoring/Alarms/Enabled` parameter to toggle Amazon CloudWatch Alarms for the cluster. <br />+  Add head node alarms to monitor Amazon EC2 health checks, CPU utilization and the overall status of the head node, and add them to the CloudWatch Dashboard created with the cluster. <br />+  Add support for Data Repository Associations when using `PERSISTENT_2` as `DeploymentType` for a managed FSx for Lustre. <br />+  Add `Scheduling/SlurmSettings/Database/DatabaseName` parameter to allow users to specify a custom name for the database on the database server to be used for Slurm accounting. <br />+  Make `InstanceType` an optional configuration parameter when configuring `CapacityReservationTarget/CapacityReservationId` in the compute resource. <br />+  Add possibility to specify a prefix for IAM roles and policies created by AWS ParallelCluster API. <br />+  Add possibility to specify a permissions boundary to be applied for IAM roles and policies created by AWS ParallelCluster API. <br />Changes+  Upgrade Slurm to 23.02.7 (from 23.02.6). <br />+  Upgrade NVIDIA driver to version 535.129.03. <br />+  Upgrade CUDA Toolkit to version 12.2.2. <br />+  Use Open Source NVIDIA GPU drivers (OpenRM) as NVIDIA kernel module for Linux instead of NVIDIA closed source module. <br />+  Remove support of `all_or_nothing_batch` configuration parameter in the Slurm resume program, in favor of the new `Scheduling/ScalingStrategy` cluster configuration. <br />+  Changed cluster alarms naming convention to '[cluster-name]-[component-name]-[metric]'. <br />+  Change default EBS volume types in AWS Secret and Top Secret regions from gp2 to gp3, for both the root and additional volumes. <br />+  The optional permissions boundary for the AWS ParallelCluster API is now applied to every IAM role created by the API infrastructure.   Upgrade EFA installer to `1.29.1`.   Efa-driver: `efa-2.6.0-1`   Efa-config: `efa-config-1.15-1`   Efa-profile: `efa-profile-1.5-1`   Libfabric-aws: `libfabric-aws-1.19.0-1`   Rdma-core: `rdma-core-46.0-1`   Open MPI: `openmpi40-aws-4.1.6-1`   <br />+  Upgrade GDRCopy to version 2.4 in all supported OSes, except for Centos 7 where version 2.3.1 is used. <br />+  Upgrade `aws-cfn-bootstrap` to version 2.0-28. <br />+  Add support for Python 3.10 in aws-parallelcluster-batch-cli. <br />Bug fixes+  Fix inconsistent scaling configuration after cluster update rollback when modifying the list of instance types declared in the Compute Resources. <br />+  Fix users SSH keys generation when switching users without root privilege in clusters integrated with an external LDAP server through cluster configuration files. <br />+  Fix disabling Slurm power save mode when setting `ScaledownIdletime = -1`. <br />+  Fix hard-coded path to Slurm installation dir in `update_slurm_database_password.sh` script for Slurm Accounting.  | December 19, 2023 | 
| AWS ParallelCluster version 3.7.2 released | AWS ParallelCluster version 3.7.2 released.<br />Changes:+  Upgrade Slurm to 23.02.6.  | October 25, 2023 | 
| AWS ParallelCluster version 3.7.1 released | AWS ParallelCluster version 3.7.1 released.<br />Changes:+  Upgrade Slurm to 23.02.5 (from 23.02.4).   Upgrade Pmix to 4.2.6 (from 3.2.3).   Upgrade libjwt to 1.15.3 (from 1.12.0).   <br />+  Upgrade EFA installer to `1.26.1`, fixing RDMA writedata issue in P5.   Efa-driver: `efa-2.5.0-1`.   Efa-config: `efa-config-1.15-1`.   Efa-profile: `efa-profile-1.5-1`.   Libfabric-aws: `libfabric-aws-1.18.2-1`.   ERdma-core: `rdma-core-46.0-1`.   Open MPI: `openmpi40-aws-4.1.5-4`.    | September 22, 2023 | 
| AWS ParallelCluster version 3.7.0 released | AWS ParallelCluster version 3.7.0 released.<br />Enhancements:+  Support configuration of static and dynamic node priorities in compute resources by using a AWS ParallelCluster configuration YAML file. <br />+  Add support for Ubuntu 22. RSA keys are not supported by default. <br />+  Add the queue configuration setting `JobExclusiveAllocation` to allocate nodes in a partition exclusively to a single job at any given time. <br />+  Allow Override `aws-parallelcluster-node` package at cluster create and cluster update time. For the head node, this applies for cluster update. Useful for development purposes only. <br />+  Avoid NFS server start on compute nodes. <br />+  Add support for log-in nodes. <br />+  Allow memory-based scheduling when multiple instance types are specified for a Slurm Compute Resource. <br />+  Add support to mount existing Amazon File Cache as shared storage. <br />Changes:+  Assign Slurm dynamic nodes a priority (weight) of 1000 by default. By doing this, Slurm can prioritize idle static nodes over idle dynamic nodes. <br />+  Make `aws-parallelcluster-node` daemons only handle AWS ParallelCluster managed Slurm partitions. <br />+  Increase `EFS-utils` watchdog poll interval to 10 seconds. This change applies when `EncryptionInTransit` is set to `true`, which is the only condition that causes the watchdog to run. <br />+  Upgrade the EFA installer to `1.25.1`.   Efa-driver: `efa-2.5.0-1` (from `efa-2.1.1g`)   Efa-config: `efa-config-1.15-1` (from `efa-config-1.13-1`)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.18.1-0` (from `libfabric-aws-1.17.1-1`)   Rdma-core: `rdma-core-46.0-1` (from `rdma-core-43.0-1`)   Open MPI: `openmpi40-aws-4.1.5-4` (from ` openmpi40-aws-4.1.5-1`)    <br />+  Upgrade Slurm to version 23.02.4. <br />+  Change the default value of Imds/ImdsSupport from v1.0 to v2.0. <br />+  Deprecate Ubuntu 18. <br />+  Update the default root volume size to 40 GB to account for limits on Centos 7. <br />+  Restrict permission on file /tmp/wait\_condition\_handle.txt within the head node so that only root can read it. <br />+  Create a Slurm partition-nodelist mapping JSON file to be used by the node package daemons to recognize PC-managed Slurm partitions and nodelists. <br />+  Upgrade NVIDIA driver to version 535.54.03. <br />+  Upgrade CUDA library to version 12.2.0. <br />+  Upgrade NVIDIA Fabric manager to nvidia-fabricmanager-535. <br />+  Upgrade ARM PL to version 23.04.1 for Ubuntu 22.04 only. <br />+  Upgrade NICE DCV to version `2023.0-15487`.   Server: `2023.0.15487-1`   xdcv: `2023.0.551-1`   gl: `2023.0.1039-1`   web\_viewer: `2023.0.15487-1`   <br />Bug fixes:+  Add validation to the `ScaledownIdletime` value, to prevent setting a value lower than -1. <br />+  Fix cluster create failure with Ubuntu Deep Learning AMI on GPU instances with DCV enabled. <br />+  Fix issue causing dangling IAM policies to be created when creating ParallelCluster CloudFormation custom resource provider with CustomLambdaRole. <br />+  Fix an issue that was causing misalignment of compute nodes DNS name on instances with multiple network interfaces, when using `SlurmSettings/Dns/UseEc2Hostnames` equals to `True` <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/releases/tag/v3.7.0), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/releases/tag/v3.7.0), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/releases/tag/v3.7.0) packages on GitHub. | August 30, 2023 | 
| Documentation only release | AWS ParallelCluster version 3 specific user guide published.<br />Documentation only release:+  AWS ParallelCluster version 3 has its own separate user guide.  | July 17, 2023 | 
| AWS ParallelCluster version 3.6.1 released | AWS ParallelCluster version 3.6.1 released.<br />Changes:+  Avoid duplication of nodes seen by `clustermgtd` if compute nodes are added to multiple Slurm partitions. <br />Bug fixes:+  Remove hard coding of root volume device name (`/dev/sda1` and `/dev/xvda`) and retrieve it from the AMIs used during `create-cluster`. <br />+  Fix cluster create failure when using CloudFormation custom resource with `ElasticIp` set to `True`. <br />+  Fix cluster create and update failures when using a AWS CloudFormation custom resource with large configuration files. <br />+  Fix an issue that prevented `ptrace` protection from being disabled on Ubuntu and that didn't permit Cross Memory Attach (CMA) in libfabric. <br />+  Fix fast insufficient capacity fail-over logic when using multiple instance types and no instances are returned. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.6.1/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.6.1/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.6.1/CHANGELOG.md) packages on GitHub. | July 5, 2023 | 
| AWS ParallelCluster version 3.6.0 released | AWS ParallelCluster version 3.6.0 released.<br />Documentation:+  Add documentation for the [AWS ParallelCluster Python library API](./pc-py-library-v3.html). <br />Enhancements:+  Add support for RHEL8. <br />+  Add an [AWS CloudFormation custom resource](./cloudformation-v3.html) for creating and managing clusters with CloudFormation. <br />+  Add support for [customizing the cluster Slurm configuration](./slurm-configuration-settings-v3.html) in the AWS ParallelCluster configuration YAML file. <br />+  Build Slurm with support for LUA. <br />+  Increase the limit on the maximum number of queues per cluster from 10 to 50. Each queue can have up to 50 compute resources. Each cluster can have up to 50 compute resources. <br />+  Add support for specifying a sequence of multiple [custom action scripts](./custom-bootstrap-actions-v3.html) for an event configured in `OnNodeStart`, `OnNodeConfigured`, and `OnNodeUpdated` parameters. <br />+  Add new configuration section `HealthChecks` / `Gpu`, for applying GPU health checks on a compute node before a job is run. <br />+  Add support for `Tags` in the `SlurmQueues` and `SlurmQueues` / `ComputeResources` configuration. <br />+  Add support for [DetailedMonitoring](./Monitoring-v3.html#yaml-Monitoring-DetailedMonitoring) in the `Monitoring` configuration. <br />+  Add `mem_used_percent` and `disk_used_percent` metrics for head node memory and root volume disk utilization tracking in the AWS ParallelCluster [CloudWatch dashboard](./cloudwatch-dashboard-v3.html), and set up alarms for monitoring these metrics. <br />+  Add [log rotation](./log-rotation-v3.html) support for AWS ParallelCluster managed logs. <br />+  Track common compute node errors and dynamic node longest idle time in the [CloudWatch Dashboard](./cloudwatch-dashboard-v3.html). <br />+  Enforce the DCV Authenticator Server to use at least `TLS-1.2` protocol when creating the SSL Socket. <br />+  Install the [NVIDIA Data Center GPU Manager (DCGM)](https://developer.nvidia.com/dcgm) package on all supported operating systems except `aarch64` `centos7` and `alinux2`. <br />+  Load the kernel module [nvidia-uvm](https://developer.nvidia.com/blog/unified-memory-cuda-beginners/) by default to provide Unified Virtual Memory (UVM) functionality to the CUDA driver. <br />+  Install the [NVIDIA Persistence Daemon](https://docs.nvidia.com/deploy/driver-persistence/index.html) as a system service. <br />Changes:+  Upgrade Slurm to version `23.02.2` (from version `22.05.8`). <br />+  Upgrade munge to version `0.5.15` (from version `0.5.14`). <br />+  Set the Slurm `TreeWidth` to 30. <br />+  Set the Slurm `prolog` and `epilog` configurations to target directory `/opt/slurm/etc/scripts/prolog.d/` and `/opt/slurm/etc/scripts/epilog.d/` respectively. <br />+  Set Slurm `BatchStartTimeout` to 3 minutes maximum for running `Prolog` scripts during compute node registration. <br />+  Increase the default `RetentionInDays` of CloudWatch logs from 14 to 180 days. <br />+  Upgrade the EFA installer to `1.22.1`.   Dkms: `2.8.3-2`   Efa-driver: `efa-2.1.1g` (no change)   Efa-config: `efa-config-1.13-1` (no change)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.17.1-1` (from `libfabric-aws-1.17.0-1`)   Rdma-core: `rdma-core-43.0-1` (no change)   Open MPI: `openmpi40-aws-4.1.5-1` (no change)   <br />+  Upgrade the Lustre client version to `2.12` on Amazon Linux 2. Lustre client `2.12` has been installed on Ubuntu 20.04, 18.04, and CentOS >= 7.7. <br />+  Upgrade the Lustre client version to `2.10.8` on CentOS 7.6. <br />+  Upgrade the NVIDIA driver to version `470.182.03` (from version `470.141.03`). <br />+  Upgrade the NVIDIA Fabric Manager to version `470.182.03` (from version `470.141.03`). <br />+  Upgrade the NVIDIA CUDA Toolkit to version `11.8.0` (from version `11.7.1`). <br />+  Upgrade the NVIDIA CUDA sample to version `11.8.0`. <br />+  Upgrade the Intel MPI Library to Version 2021 Update 9 (from Version 2021 Update 6). For more information, see [Intel® MPI Library 2021 Update 9](https://software.intel.com/content/www/us/en/develop/articles/intel-mpi-library-release-notes-linux.html#inpage-nav-3). <br />+  Upgrade NICE DCV to version `2023.0-15022` (from version `2022.2-14521`).   server: `2023.0.15022-1` (from version `2022.2-14521-1`).   xdcv: `2023.0.547-1` (from version `2022.2.519-1`).   gl: `2023.0.1027-1` (from version `2022.2.1012-1`).   web\_viewer: `2023.0.15022-1` (from version `2022.2.14521-1`).   <br />+  Upgrade `aws-cfn-bootstrap` to version `2.0-24`. <br />+  Upgrade image used by the CodeBuild environment when building container images for AWS Batch clusters:   `aws/codebuild/amazonlinux2-x86_64-standard:4.0` (from `aws/codebuild/amazonlinux2-x86_64-standard:3.0`).   `aws/codebuild/amazonlinux2-aarch64-standard:2.0` (from `aws/codebuild/amazonlinux2-aarch64-standard:1.0`).   <br />Bug fixes:+  Fix Amazon EFS and Amazon FSx network security group validators to avoid reporting false errors. <br />+  Fix missing tagging of resources created by Image Builder during the `build-image` operation. <br />+  Fix update policy for `MaxCount` to always perform numerical comparisons on the `MaxCount` property. <br />+  Fix IP alignment on compute node instances with multiple network cards. <br />+  Fix replacement of `StoragePass` in the `slurm_parallelcluster_slurmdbd.conf` when a queue parameter update is performed and the Slurm accounting configurations are not updated. <br />+  Fix issue that causes dangling security groups to be created when creating a cluster with an existing EFS file system. <br />+  Fix issue causing the `cfn-hup` daemon to fail when it gets restarted. <br />+  Consider dynamic nodes with `INVALID_REG` flag as bootstrap failures for Slurm protected mode. Static nodes failing Slurm registration are already treated as bootstrap failures after the `node_replacement_timeout`. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.6.0/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.6.0/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.6.0/CHANGELOG.md) packages on GitHub. | May 22, 2023 | 
| AWS ParallelCluster version 3.5.1 released | AWS ParallelCluster version 3.5.1 released.<br />Enhancements:+  Add a stand-alone `pcluster` CLI [installer executable](./install-v3-install-standalone.html). <br />Changes:+  Upgrade EFA installer to `1.22.0`.   Efa-driver: `efa-2.1.1g` (from `efa-2.1.1-1`)   Efa-config: `efa-config-1.13-1` (from efa-config-1.12-1)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.17.0-1` (from `libfabric-aws-1.16.1amzn3.0-1`)   Rdma-core: `rdma-core-43.0-1` (no change)   Open MPI: `openmpi40-aws-4.1.5-1` (from `openmpi40-aws-4.1.4-3`)   <br />Upgrade NICE DCV to version `2022.2-14521`.   server: `2022.2.14521-1`   xdcv: `2022.2.519-1`   gl: `2022.2.1012-1`   web\_viewer: `2022.2.14521-1`   <br />Bug fixes:+  Fix potential node launch failures caused by pattern matching between `MountDir` and `/etc/exports` when removing shared Amazon EBS volumes as part of a cluster update. <br />+  Fix to prevent `compute_console_output` log file truncation at every `clustermgtd` iteration. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.5.1/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.5.1/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.5.1/CHANGELOG.md) packages on GitHub. | March 29, 2023 | 
| AWS ParallelCluster version 3.5.0 released | AWS ParallelCluster version 3.5.0 released.<br />Enhancements:+  Access and manage clusters with the [AWS ParallelCluster UI](./pcui-using-v3.html). <br />+  Add versioned AWS ParallelCluster policies in a CloudFormation template that you can reference in your workloads. <br />+  Add a AWS ParallelCluster Python library that you can use with your own code. <br />+  Add logging of compute node console output to Amazon CloudWatch on compute node bootstrap failure. <br />+  Add failures field containing failure code and reason to `describe-cluster` output when cluster creation fails. <br />+  Add validators to prevent malicious string injection while calling the subprocess module. <br />+  Fail cluster creation if cluster status changes to `PROTECTED` while provisioning static nodes. <br />Changes:+  Upgrade to Slurm version `22.05.8` (from version `22.05.7`) <br />+  Upgrade EFA installer to `1.21.0`.   Efa-driver: `efa-2.1.1-1` (from `efa-2.1`)   Efa-config: `efa-config-1.12-1` (from efa-config-1.11-1)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.16.1amzn3.0-1` (from `libfabric-aws-1.16.1`)   Rdma-core: `rdma-core-43.0-1` (from `rdma-core-43.0-2`)   Open MPI: `openmpi40-aws-4.1.4-3` (no change)   <br />+  Make Slurm controller logs more verbose and enable additional logging for the Slurm power save plugin. <br />Bug fixes:+  Fix cluster database creation by verifying that the cluster name is not longer than 40 characters when Slurm accounting is enabled. <br />+  Fix an issue in `clustermgtd` that caused compute nodes, rebooted through Slurm, to be replaced if the Amazon EC2 instance status checks fail. <br />+  Fix an issue that prevented compute nodes, with capacity reservations shared by other accounts, from launching because of an incorrect IAM policy on the head node. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.5.0/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.5.0/CHANGELOG.md), [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.5.0/CHANGELOG.md), and [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.02) packages on GitHub. | February 20, 2023 | 
| AWS ParallelCluster version 3.4.1 released | AWS ParallelCluster version 3.4.1 released.<br />Bug fixes:+  Fix a Slurm scheduler issue that could cause the incorrect application of updates to its internal registry of compute nodes. As a result if this issue, EC2 instances could become unavailable or could be backed by an incorrect instance type. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.4.1/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.4.1/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.4.1/CHANGELOG.md) packages on GitHub. | January 13, 2023 | 
| AWS ParallelCluster version 3.4.0 released | AWS ParallelCluster version 3.4.0 released.<br />Enhancements:+  Add support for launching nodes across multiple availability zones to increase capacity availability. <br />+  Add support for specifying multiple subnets for each queue to increase capacity availability. <br />+  Add new configuration parameter in [Iam](./Iam-v3.html) / [ResourcePrefix](./Iam-v3.html#yaml-Iam-ResourcePrefix) to specify a prefix for path and name of IAM resources created by AWS ParallelCluster. <br />+  Add new configuration section [DeploymentSettings](./DeploymentSettings-cluster-v3.html) / [LambdaFunctionsVpcConfig](./DeploymentSettings-cluster-v3.html#DeploymentSettings-cluster-v3-LambdaFunctionsVpcConfig) for specifying the Vpc config used by AWS ParallelCluster Lambda functions. <br />+  Add the ability to specify a custom script to run in the head node during a cluster update. The script can be specified with [HeadNode](./HeadNode-v3.html) / [CustomActions](./HeadNode-v3.html#HeadNode-v3-CustomActions) / [OnNodeUpdated](./HeadNode-v3.html#yaml-HeadNode-CustomActions-OnNodeUpdated) when using Slurm as scheduler. <br />Changes:+  Remove creation of Amazon EFS mount targets for existing file systems. <br />+  Mount EFS file systems using `amazon-efs-utils`. EFS files systems can be mounted using in-transit encryption and an IAM authorized user. <br />+  Install stunnel 5.67 on CentOS7 and Ubuntu to support EFS in-transit encryption. <br />+  Upgrade EFA installer to `1.20.0` (from `1.18.0`).   Efa-driver: `efa-2.1` (from `efa-1.16.0-1`)   Efa-config: `efa-config-1.11-1` (no change)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.16.1` (from `libfabric-aws-1.16.0~amzn4.0-1`)   Rdma-core: `rdma-core-43.0-2` from (`rdma-core-41.0-2`)   Open MPI: `openmpi40-aws-4.1.4-3` from (`openmpi40-aws-4.1.4-2`)   <br />+  Upgrade Slurm to version `22.05.7` (from `22.05.5`). <br />+  Upgrade Python to `3.9.16` and `3.7.16`. (from `3.9.15` and `3.7.13`). <br />+  With Slurm `22.05.7`, dynamic nodes in `IDLE+CLOUD+COMPLETING+POWER_DOWN+NOT_RESPONDING` status aren't considered unhealthy. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.4.0/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.4.0/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.4.0/CHANGELOG.md) packages on GitHub. | December 22, 2022 | 
| AWS ParallelCluster version 3.3.1 released | AWS ParallelCluster version 3.3.1 released.<br />Changes:+  Official AWS ParallelCluster product AMIs are now available after Amazon EC2 deprecation at two years. <br />+  Increase memory size of the AWS ParallelCluster API Lambda to 2048 in order to reduce cold start penalties and avoid timeouts. <br />Bug fixes:+  Prevent replacement of managed FSx for Lustre file systems and loss of data on cluster updates that include changes to the compute fleet subnet ID. <br />+  [SharedStorage](./SharedStorage-v3.html) `DeletionPolicy` applies to cluster update actions. <br />For details of the changes, see the `CHANGELOG` file for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.3.1/CHANGELOG.md) package on GitHub. | December 2, 2022 | 
| AWS ParallelCluster documentation only hpc6id note | AWS ParallelCluster documentation-only update+  AWS ParallelCluster doesn't support the hpc6id instance type for the [HeadNode](./HeadNode-v3.html) / [InstanceType](./HeadNode-v3.html#yaml-HeadNode-InstanceType) setting.  | December 2, 2022 | 
| AWS ParallelCluster version 3.1.5 released | AWS ParallelCluster version 3.1.5 released.<br />Enhancements:+  Fix Slurm issue that prevents idle nodes termination. <br />+  Upgrade EFA installer to 1.18.0   Efa-driver: `efa-1.16.0-1`   Efa-config: `efa-config-1.11-1` (from `efa-config-1.9-1`)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.16.0~amzn4.0-1` (from `libfabric-1.13.2`).   Rdma-core: `rdma-core-41.0-2` (from `rdma-core-37.0`)   Open MPI: `openmpi40-aws-4.1.4-2` (from `openmpi40-aws-4.1.1-2`)   <br />Changes:+  Add `lambda:ListTags` and `lambda:UntagResource` to the `ParallelClusterUserRole` used by the AWS ParallelCluster API stack for a cluster update. <br />+  Upgrade Intel MPI Library to Version 2021 Update 6 (from Version 2021 Update 4). For more information, see [Intel® MPI Library 2021 Update 6](https://software.intel.com/content/www/us/en/develop/articles/intel-mpi-library-release-notes-linux.html#inpage-nav-3-4). <br />+  Upgrade NVIDIA driver to version 470.141.03 (from 470.103.01). <br />+  Upgrade NVIDIA Fabric Manager to version 470.141.03 (from 470.103.01). <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.1.5/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.1.5/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.1.5/CHANGELOG.md) packages on GitHub. | November 16, 2022 | 
| AWS ParallelCluster version 3.3.0 released | AWS ParallelCluster version 3.3.0 released.<br />Enhancements:+  Add support for multiple instance allocation configuration for a compute resource when using Slurm as a scheduler. For more information, see [Multiple instance type allocation with Slurm](./slurm-multiple-instance-allocation-v3.html). <br />+  Add support for adding and removing [SharedStorage](./SharedStorage-v3.html) with a cluster update, using an updated configuration. For more information, see [Shared storage](./shared-storage-quotas-integration-v3.html). <br />+  Add new configuration parameter `DeletionPolicy` for [Efs](./SharedStorage-v3.html#SharedStorage-v3-EfsSettings) and [FsxLustre](./SharedStorage-v3.html#SharedStorage-v3-FsxLustreSettings) shared storage settings to support storage retention. <br />+  Add support for Slurm accounting with new configuration parameter [Scheduling](./Scheduling-v3.html) / [SlurmSettings](./Scheduling-v3.html#Scheduling-v3-SlurmSettings) / [Database](./Scheduling-v3.html#Scheduling-v3-SlurmSettings-Database). For more information, see [Slurm accounting with AWS ParallelCluster](./slurm-accounting-v3.html). <br />+  Add support for On-Demand Capacity Reservations (ODCR) and capacity reservation resource groups. For more information, see [Launch instances with On-Demand Capacity Reservations (ODCR)](./launch-instances-odcr-v3.html). <br />+  Add new configuration parameter to specify the IMDS version to support in a cluster or build image infrastructure in the cluster, [Imds](./Imds-cluster-v3.html) / [ImdsSupport](./Imds-cluster-v3.html#yaml-cluster-Imds-ImdsSupport), and build, [Imds](./Build-v3.html#Build-v3-Imds) / [ImdsSupport](./Build-v3.html#yaml-build-image-Build-Imds-ImdsSupport), configurations. <br />+  Add support for [Networking](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-ComputeResources-Networking) / [PlacementGroup](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-ComputeResources-Networking-PlacementGroup) in the [SlurmQueues](./Scheduling-v3.html#Scheduling-v3-SlurmQueues) / [ComputeResources](./Scheduling-v3.html#Scheduling-v3-SlurmQueues-ComputeResources) section. <br />+  Add support for instances with multiple network interfaces that are limited to only one ENI per device. <br />+  Improve validation of networking for external Amazon EFS file systems by checking the CIDR block in the attached security group. <br />+  Add validator to check if configured instance types support placement groups. <br />+  Configure NFS threads to be min(256, max(8, num\_cores \* 4)) to ensure better stability and performance. <br />+  Move NFS installation at build time to reduce configuration time. <br />+  Enable server-side encryption for the EcrImageBuilder SNS topic that's created when deploying AWS ParallelCluster API and is used to notify on docker image build events. <br />Changes:+  Change the behavior of [SlurmQueues](./Scheduling-v3.html#Scheduling-v3-SlurmQueues) / [Networking](./Scheduling-v3.html#Scheduling-v3-SlurmQueues-Networking) / [PlacementGroup](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup) / `Enabled`. It now creates a unique managed placement group for each compute resource instead of a single managed placement group for all compute resources. <br />+  Add support for [SlurmQueues](./Scheduling-v3.html#Scheduling-v3-SlurmQueues) / [Networking](./Scheduling-v3.html#Scheduling-v3-SlurmQueues-Networking) / [PlacementGroup](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup) / [Name](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup) as the preferred naming method. <br />+  Move head node tags from Launch Template to instance definition to avoid head node replacement on tags updates. <br />+  Disable multithreading through script executed by `cloud-init` and not through `CpuOptions` set in the Launch Template. <br />+  Upgrade Python to version 3.9 and NodeJS to version 16 in the API infrastructure, API Docker container, and cluster Lambda resources. <br />+  Remove support for Python 3.6 in `aws-parallelcluster-batch-cli`. <br />+  Upgrade Slurm to version `22.05.5` (from `21.08.8-2`). <br />+  Upgrade NVIDIA driver to version `470.141.03` (from `470.129.06`). <br />+  Upgrade NVIDIA Fabric Manager to version `470.141.03` (from `470.129.06`). <br />+  Upgrade NVIDIA CUDA Toolkit to version 11.7.1 (`from 11.4.4`). <br />+  Upgrade Python used in AWS ParallelCluster virtualenvs from `3.7.13` to `3.9.15`. <br />+  Upgrade EFA installer to version 1.18.0.   Efa-driver: `efa-1.16.0-1` (no change)   Efa-config: `efa-config-1.11-1` (`from efa-config-1.10-1`)   Efa-profile: `efa-profile-1.5-1` (no change)   Libfabric-aws: `libfabric-aws-1.16.0~amzn4.0-1` (from `libfabric-aws-1.16.0~amzn2.0-1`).   Rdma-core: `rdma-core-41.0-2` (from `rdma-core-37.0`)   Open MPI: `openmpi40-aws-4.1.4-2` (from `openmpi40-aws-4.1.1-2`)   <br />+  Upgrade NICE DCV to version `2022.1-13300` (from `2022.0-12760`). <br />+  Enable suppression of the `SingleSubnetValidator` for `Queues`. <br />+  Do not replace `DRAIN` nodes when nodes are in `COMPLETING` state as Epilog may be still running. <br />Bug fixes:+  Fix validation of filters parameter in the AWS ParallelCluster `ListClusterLogStreams` command to fail when incorrect filters are passed. <br />+  Fix validation of parameter [SharedStorage](./SharedStorage-v3.html) / [EfsSettings](./SharedStorage-v3.html#SharedStorage-v3-EfsSettings) to fail validation when `FileSystemId` is specified along with other [SharedStorage](./SharedStorage-v3.html) / [EfsSettings](./SharedStorage-v3.html#SharedStorage-v3-EfsSettings) parameters. Previously, `FileSystemId` wasn't included. <br />+  Fix cluster update when changing the order of [SharedStorage](./SharedStorage-v3.html) together with other changes in the configuration. <br />+  Fix `UpdateParallelClusterLambdaRole` in the AWS ParallelCluster API to upload logs to CloudWatch. <br />+  Fix Cinc not using the local CA certificates bundle when installing packages before any cookbooks are executed. <br />+  Fix a hang in upgrading ubuntu with `pcluster build-image` when `Build:UpdateOsPackages:Enabled:true` is set. <br />+  Fix parsing of YAML cluster configuration by failing on duplicate keys. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.3.0/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.3.0/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.3.0/CHANGELOG.md) packages on GitHub. | November 2, 2022 | 
| AWS ParallelCluster documentation only API reference added. | AWS ParallelCluster documentation-only update+  Added the version 3 [AWS ParallelCluster API reference](./api-ref-v3.html) to the documentation.  | October 27, 2022 | 
| AWS ParallelCluster version 3.2.1 released | AWS ParallelCluster version 3.2.1 released.<br />Enhancements:+  Improve the logic to associate the host routing tables to the different network cards to better support Amazon EC2 instances with several NICs. <br />Changes:+  Upgrade NVIDIA driver to version 470.141.03. <br />+  Upgrade NVIDIA Fabric Manager to version 470.141.03. <br />+  Disable `cron` job tasks `man-db` and `mlocate`, which may have a negative impact on node performance. <br />+  Upgrade Intel MPI Library to 2021.6.0.602. <br />+  Upgrade Python from 3.7.10 to 3.7.13 in response to this security risk. <br />Bug fixes:+  Avoid failing on `DescribeCluster` when cluster configuration is not available. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.2.1/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.2.1/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.2.1/CHANGELOG.md) packages on GitHub. | October 3, 2022 | 
| AWS ParallelCluster version 3.2.0 released | AWS ParallelCluster version 3.2.0 released.<br />Enhancements:+  Add support for [memory-based scheduling](./slurm-mem-based-scheduling-v3.html) in Slurm.   Configure compute nodes real memory in the Slurm cluster configuration.   Add new configuration parameter [Scheduling](./Scheduling-v3.html) / [SlurmSettings](./Scheduling-v3.html#Scheduling-v3-SlurmSettings) / [EnableMemoryBasedScheduling](./Scheduling-v3.html#yaml-Scheduling-SlurmSettings-EnableMemoryBasedScheduling) to enable memory-based scheduling in Slurm.   Add new configuration parameter [Scheduling](./Scheduling-v3.html) / [SlurmQueues](./Scheduling-v3.html#Scheduling-v3-SlurmQueues) / [ComputeResources](./Scheduling-v3.html#Scheduling-v3-SlurmQueues-ComputeResources) / [SchedulableMemory](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-ComputeResources-SchedulableMemory) to override the default value of the memory seen by the scheduler on compute nodes.   <br />+  Improve flexibility on cluster configuration updates to avoid the stop and start of the entire cluster whenever possible. Add new configuration parameter [Scheduling](./Scheduling-v3.html) / [SlurmSettings](./Scheduling-v3.html#Scheduling-v3-SlurmSettings) / [QueueUpdateStrategy](./Scheduling-v3.html#yaml-Scheduling-SlurmSettings-QueueUpdateStrategy) to set the preferred strategy to use when compute nodes need a configuration update and replacement. <br />+  Improve fail-over mechanism over available compute resources when hitting insufficient capacity issues with Amazon EC2 instances. [Disable compute nodes by a configurable amount of time](./slurm-short-capacity-fail-mode-v3.html) when a node launch fails due to insufficient capacity. <br />+  Add support to mount existing [FSx for ONTAP](./SharedStorage-v3.html#SharedStorage-v3-FsxOntapSettings) and [FSx for OpenZFS](./SharedStorage-v3.html#SharedStorage-v3-FsxOpenZfsSettings) file systems. <br />+  Add support to mount multiple instances of existing [Amazon Elastic File Systems, FSx for Lustre, FSx for ONTAP, and FSx for OpenZFS](./SharedStorage-v3.html) file systems. <br />+  Add support for [FSx for Lustre Persistent\_2 deployment type](./SharedStorage-v3.html#yaml-SharedStorage-FsxLustreSettings-DeploymentType) when creating a new file system. <br />+  Prompt user to enable EFA for supported instance types when using the [pcluster configure](./pcluster.configure-v3.html) wizard. <br />+  Add support for rebooting compute nodes using Slurm. <br />+  Improve handling of Slurm power states to also account for manual powering down of nodes. <br />+  Install NVIDIA GDRCopy 2.3 into the product AMIs to enable low-latency GPU memory copy. <br />Changes:+  Upgrade EFA installer to version 1.17.2.   EFA driver: `efa-1.16.0-1`   EFA configuration: `efa-config-1.10-1`   EFA profile: `efa-profile-1.5-1`   Libfabric: `libfabric-aws-1.16.0~amzn2.0-1`   RDMA core: `rdma-core-41.0-2`   Open MPI: `openmpi40-aws-4.1.4-2`   <br />+  Upgrade NICE DCV to version 2022.0-12760. <br />+  Upgrade NVIDIA driver to version 470.129.06. <br />+  Upgrade NVIDIA Fabric Manager to version 470.129.06. <br />+  Change default EBS volume types from gp2 to gp3 in both the root and additional volumes. <br />+  Changes to FSx for Lustre file systems created by AWS ParallelCluster:   Change the default deployment type to `Scratch_2`.   Change the Lustre server version to `2.12`.   <br />+  Doesn't require [PlacementGroup](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup) / [Enabled](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup-Enabled) to be set to `true` when passing an existing `PlacementGroup` / `Id`. <br />+  Doesn't allow setting `PlacementGroup` / `Id` when `PlacementGroup` / `Enabled` is explicitly set to `false`. <br />+  Add `parallelcluster:cluster-name` tag to all resources created by AWS ParallelCluster. <br />+  Add `lambda:ListTags` and `lambda:UntagResource` to `ParallelClusterUserRole` used by the AWS ParallelCluster API stack for cluster update. <br />+  Restrict IPv6 access to `IMDS` to root and cluster admin users only, when configuration parameter `HeadNode` / `Imds` / `Secured` is enabled. <br />+  With a custom AMI, use the AMI root volume size instead of the ParallelCluster default of 35 GiB. The value can be changed in cluster configuration file. <br />+  Automatic disabling of the compute fleet when the configuration parameter `Scheduling` / `SlurmQueues` / `ComputeResources` / `SpotPrice` is lower than the minimum required Spot request fulfillment price. <br />+  Show `requested_value` and `current_value` values in the change set when adding or removing a section during an update. <br />+  Disable `aws-ubuntu-eni-helper` service, available in Deep Learning AMIs, to avoid conflicts with `configure_nw_interface.sh` when configuring instances with multiple network cards. <br />+  Remove support for Python 3.6. <br />+  Set MTU to 9001 for all the network interfaces when configuring instances with multiple network cards. <br />+  Remove the trailing dot when configuring the compute node FQDN. <br />+  Manage static nodes in `POWERING_DOWN`. <br />+  Doesn't replace dynamic node in `POWER_DOWN` as jobs may be still running. <br />+  Restart `clustermgtd` and `slurmctld` daemons at cluster update time only when `Scheduling` parameters are updated in the cluster configuration. <br />+  Update `slurmctld` and `slurmd` `systemd` service files. <br />+  Restrict IPv6 access to IMDS to root and cluster admin users only, when configuration parameter `HeadNode` / `Imds` / `Secured` is enabled. <br />+  Set Slurm configuration `AuthInfo=cred_expire=70` to reduce the time requeued jobs must wait before starting again when nodes are not available. <br />+  Upgrade third-party cookbook dependencies:   apt-7.4.2 (from apt-7.4.0)   line-4.5.2 (from line-4.0.1)   openssh-2.10.3 (from openssh-2.9.1)   pyenv-3.5.1 (from pyenv-3.4.2)   selinux-6.0.4 (from selinux-3.1.1)   yum-7.4.0 (from yum-6.1.1)   yum-epel-4.5.0 (from yum-epel-4.1.2)   <br />Bug fixes:+  Fix the default behavior to skip the AWS ParallelCluster validation and test steps when building a custom AMI. <br />+  Fix file handle leak in `computemgtd`. <br />+  Fix race condition that was sporadically causing launched instances to be immediately terminated because they were not yet available in the EC2 DescribeInstances response. <br />+  Fix support for the `DisableSimultaneousMultithreading` parameter on instance types with Arm processors. <br />+  Fix AWS ParallelCluster API stack update failure when upgrading from a previous version. Add resource pattern used for the `ListImagePipelineImages` Action in the `EcrImageDeletionLambdaRole`. <br />+  Fix AWS ParallelCluster API adding missing permissions needed to import or export from Amazon S3 when creating an FSx for Lustre file system. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.2.0/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.2.0/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.2.0/CHANGELOG.md) packages on GitHub. | July 27, 2022 | 
| AWS ParallelCluster documentation-only updates this year to date | AWS ParallelCluster documentation-only updates.<br />New sections:+  [Best practices: budget alerts](./best-practices-v3.html#best-practices-budget-alerts-v3) V3 <br />+  [Best practices: moving a cluster to a new AWS ParallelCluster minor or patch version](./best-practices-v3.html#best-practices-cluster-upgrades-v3) V3 <br />+  [Working with Amazon S3](./s3_resources-v3.html) V3 <br />+  [Working with Spot Instances](./spot-v3.html) V3 <br />+  [Slurm cluster protected mode](./slurm-protected-mode-v3.html) V3 <br />+  [AWS ParallelCluster resources and tagging](./resources-tags-v3.html) V3 <br />+  [Amazon CloudWatch dashboard](./cloudwatch-dashboard-v3.html) V3 <br />+  [Integration with Amazon CloudWatch Logs](./cloudwatch-logs-v3.html) V3 <br />+  [Elastic Fabric Adapter](./efa-v3.html) V3 <br />+  [AWS ParallelCluster AMI customization](./custom-ami-v3.html) V3 <br />+  [Launch instances with On-Demand Capacity Reservations (ODCR)](./launch-instances-odcr-v3.html) V3 <br />+  [AMI patching and Amazon EC2 instance replacement](./instance-updates-ami-patch-v3.html) V3 <br />+  [How AWS ParallelCluster works](./functional-v3.html) V3 <br />+  [Configuring shared storage encryption with an AWS KMS key](./tutorials_04_encrypted_kms_fs-v3.html) V3 <br />+  [Running jobs in a multiple queue mode cluster](./multi-queue-tutorial-v3.html) V3 <br />+  [Using the AWS ParallelCluster API](./tutorials_06_API_use.html) V3 <br />Section updates:+  [Best practices: network performance](./best-practices-v3.html#best-practices-network-performance-v3) V3: Added best practices for using Elastic Fabric Adaptor. <br />+  [AWS Identity and Access Management permissions in AWS ParallelCluster](./iam-roles-in-parallelcluster-v3.html) V3: Various updates and added [Additional AWS ParallelCluster pcluster user policy when using Amazon FSx for Lustre](./iam-roles-in-parallelcluster-v3.html#iam-roles-in-parallelcluster-v3-user-policy-fsxlustre). <br />+  [AWS ParallelCluster troubleshooting](./troubleshooting-v3.html) V3: Various updates.  | July 6, 2022 | 
| AWS ParallelCluster version 3.1.4 released | AWS ParallelCluster version 3.1.4 released.<br />Enhancements:+  Add validation for [DirectoryService](./DirectoryService-v3.html) / [PasswordSecretArn](./DirectoryService-v3.html#yaml-DirectoryService-PasswordSecretArn) to fail if the secret doesn't exist. <br />Add support for enabling JWT authentication Slurm. <br />Changes:+  Upgrade Slurm to version 21.08.8-2. <br />+  Build Slurm with JWT support. <br />+  Doesn't require [PlacementGroup](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup) / [Enabled](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-Networking-PlacementGroup-Enabled) to be set to `true` when passing an existing `PlacementGroup` / `Id`. <br />+  Add `lambda:TagResource` to `ParallelClusterUserRole` used by ParallelCluster API stack for cluster creation and image creation. <br />Bug fixes:+  Fix the ability to export a cluster's logs when using the `export-cluster-logs` command with the `--filters` option. <br />+  Fix AWS Batch Docker entry point to use `/home` shared directory to coordinate Multi-node-Parallel job execution. <br />+  Reset node address when setting Slurm unhealthy static node to down to avoid treating static node failed with insufficient capacity as a bootstrap failure node. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.1.4/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.1.4/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.1.4/CHANGELOG.md) packages on GitHub. | May 16, 2022 | 
| AWS ParallelCluster version 3.1.3 released | AWS ParallelCluster version 3.1.3 released.<br />Enhancements:+  Execute SSH key creation alongside with the creation of HOME directory, for example, during SSH login, when switching to another user and when executing a command as another user. <br />+  Add support for both FQDN and LDAP Distinguished Names in the configuration parameter [DirectoryService](./DirectoryService-v3.html) / [DomainName](./DirectoryService-v3.html#yaml-DirectoryService-DomainName). The new validator now checks both the syntaxes. <br />+  New `update_directory_service_password.sh` script deployed on the head node supports the manual update of the Active Directory password in the SSSD configuration. The password is retrieved by the AWS Secrets Manager as from the cluster configuration. <br />+  Add support to deploy API infrastructure in environments without a default VPC. <br />Changes:+  Disable deeper C-States in x86\_64 official AMIs and AMIs created through `build-image` command, to guarantee high performance and low latency. <br />+  OS package updates and security fixes. <br />+  Change Amazon Linux 2 base images to use AMIs with Kernel 5.10. <br />Bug fixes:+  Fix build-image stack in `DELETE_FAILED` after image built successful, due to new EC2 Image Builder policies. <br />+  Fix the configuration parameter [DirectoryService](./DirectoryService-v3.html) / [DomainAddr](./DirectoryService-v3.html#yaml-DirectoryService-DomainAddr) conversion to ldap\_uri SSSD property when it contains multiples domain addresses. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.1.3/CHANGELOG.md), and [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.1.3/CHANGELOG.md) packages on GitHub. | April 20, 2022 | 
| AWS ParallelCluster version 3.1.2 released | AWS ParallelCluster version 3.1.2 released.<br />Changes:+  Upgrade Slurm to version `21.08.6` (from `21.08.5`). <br />Bug fixes:+  Fix the update of `/etc/hosts` file on compute nodes when a cluster is deployed in subnets without internet access. <br />+  Fix compute nodes bootstrap to wait for ephemeral drives initialization before joining the cluster. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.1.2/CHANGELOG.md) package on GitHub. | March 2, 2022 | 
| AWS ParallelCluster version 3.1.1 released | AWS ParallelCluster version 3.1.1 released.+  Add support for multiple user cluster environments by [integrating with Active Directory (AD) domains](./DirectoryService-v3.html) managed through AWS Directory Service. <br />+  Add support for [UseEc2Hostnames](./Scheduling-v3.html#Scheduling-v3-SlurmSettings) in the cluster configuration file. When set to true, use Amazon EC2 default hostnames (e.g. ip-1-2-3-4) for compute nodes. <br />+  Add support for cluster creation in [subnets with no internet access](./aws-parallelcluster-in-a-single-public-subnet-no-internet-v3.html). <br />+  Add support for multiple compute instance types per queue. <br />+  Add support for GPU scheduling with Slurm on ARM instances with NVIDIA cards. <br />+  Add abbreviated flags for `cluster-name` (`-n`), `region` (`-r`), `image-id` (`-i`) and `cluster-configuration` / `image-configuration` (`-c`) to the AWS ParallelCluster CLI. <br />+  Add support for `NEW_CHANGED_DELETED` option for FSx for Lustre [AutoImportPolicy](./SharedStorage-v3.html#yaml-SharedStorage-FsxLustreSettings-AutoImportPolicy) parameter. <br />+  Add `parallelcluster:compute-resource-name` tag to EC2 `LaunchTemplates` resources used by compute nodes. <br />+  Improve security groups created within the cluster to allow inbound connections from custom security groups when `SecurityGroups` parameters are specified for some head node and/or queues. <br />+  Install NVIDIA drivers and CUDA library for ARM. <br />Changes:+  Upgrade Slurm to version `21.08.5` (from `20.11.8`). <br />+  Upgrade Slurm plugin to version `21.08` (from `20.11`). <br />+  Upgrade NICE DCV to version `2021.3-11591` (from `2021.1-10851`). <br />+  Upgrade NVIDIA driver to version `470.103.01` (from `470.57.02`). <br />+  Upgrade NVIDIA Fabric manager to version `470.103.01` (from `470.57.02`). <br />+  Upgrade CUDA to version `11.4.4` (from `11.4.0`). <br />+  [Intel MPI](./intelmpi-v3.html) updated to Version 2021 Update 4 (updated from Version 2019 Update 8). For more information, see [Intel® MPI Library 2021 Update 4](https://software.intel.com/content/www/us/en/develop/articles/intel-mpi-library-release-notes-linux.html#inpage-nav-3-1). <br />+  Upgrade PMIx to version `3.2.3` (from `3.1.5`). <br />+  Remove dumping of failed compute nodes to `/home/logs/compute`. Compute nodes log files are available in CloudWatch and in Amazon EC2 console logs. <br />+  Enable potential to suppress `SlurmQueues` and `ComputeResources` length validators. <br />+  Disable package update at instance launch time on Amazon Linux 2. <br />+  Disable Amazon EC2 ImageBuilder enhanced image metadata when building AWS ParallelCluster custom images. <br />+  Explicitly set `cloud-init` datasource to be EC2. This saves boot time for Ubuntu and CentOS platforms. <br />+  Use compute resource name rather than instance type in compute fleet launch template name. <br />+  Redirect stderr and stdout to CLI log file to prevent unwanted text in the pcluster CLI output. <br />+  Move the configure/install recipes to separate cookbooks that are called from the main one. Existing entrypoints are maintained and backwards compatible. <br />+  Download dependencies of Intel HPC platform during AMI build time to avoid contacting internet during cluster creation time. <br />+  Do not strip `-` from compute resource name when configuring Slurm nodes. <br />+  Do not configure GPUs in Slurm when NVIDIA driver is not installed. <br />+  Fix `ecs:ListContainerInstances` permission in `BatchUserRole`. <br />+  Fix exporting of cluster logs when there is no prefix specified, previously exported to a `None` prefix. <br />+  Fix rollback not being performed in case of cluster update failure. <br />+  Fix `ecs:ListContainerInstances` permission in `BatchUserRole`. <br />+  Fix `RootVolume` schema for the `HeadNode` by raising an error if an unsupported `KmsKeyId` is specified. <br />+  Fix Amazon FSx missing metrics to be displayed in CloudWatch Dashboard. <br />+  Fix `EfaSecurityGroupValidator`. Previously, it had potential to produce false failures when custom security groups were provided and EFA was enabled. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.1.1/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.1.1/CHANGELOG.md), and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.1.1/CHANGELOG.md) packages on GitHub. | February 10, 2022 | 
| AWS ParallelCluster version 3.0.3 released | AWS ParallelCluster version 3.0.3 released.+  Disable `log4j-cve-2021-44228-hotpatch` agent (`Log4jHotPatch`) on Amazon Linux 2 to avoid potential performance degradation. For more information, see [Amazon Linux Hotpatch Announcement for Apache Log4j](https://alas.aws.amazon.com/announcements/2021-001.html). <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v2.11.4/CHANGELOG.md) and [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v2.11.4/CHANGELOG.md) packages on GitHub. | January 17, 2022 | 
| AWS ParallelCluster version 3.0.2 released | AWS ParallelCluster version 3.0.2 released.<br />Upgrade [Elastic Fabric Adapter](./efa-v3.html) installer to 1.14.1+  EFA config: `efa-config-1.9-1` (from `efa-config-1.9`) <br />+  EFA profile: `efa-profile-1.5-1` (from `efa-profile-1.5`) <br />+  EFA Kernel module: `efa-1.14.2` (from `efa-1.13.0`) <br />+  RDMA core: `rdma-core-37.0` (from `rdma-core-35`) <br />+  Libfabric: `libfabric-1.13.2` (from `libfabric-1.13.0`) <br />+  Open MPI: `openmpi40-aws-4.1.1-2` (no change) <br />GPUDirect RDMA is always enabled if supported by the instance type. The [GdrSupport](./Scheduling-v3.html#yaml-Scheduling-SlurmQueues-ComputeResources-Efa-GdrSupport) configuration option has no effect.<br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.0.2/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.0.2/CHANGELOG.md) and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.0.2/CHANGELOG.md) packages on GitHub. | November 5, 2021 | 
| AWS ParallelCluster version 3.0.1 released | AWS ParallelCluster version 3.0.1 released.<br />Cluster configuration migration tool+  Customers can now migrate their cluster configurations from the AWS ParallelCluster version 2 format to the YAML-based AWS ParallelCluster version 3 format. For more information, see [pcluster3-config-converter](./pcluster3-config-converter.html). <br />Head node can be stopped+  After stopping the compute fleet, the head node can be stopped and later restarted using the Amazon EC2 console or the [stop-instances](https://docs.aws.amazon.com/cli/latest/reference/ec2/stop-instances.html) AWS CLI command. <br />Default AWS Region read from `~/.aws/config` file+  For the [pcluster](./pcluster-v3.html) command, if the AWS Region is not specified in the configuration file, in the environment, or on the command line, the default AWS Region specified in the `region` setting in the `[default]` section of the `~/.aws/config` file is used. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.0.1/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.0.1/CHANGELOG.md) and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.0.1/CHANGELOG.md) packages on GitHub. | October 27, 2021 | 
| AWS ParallelCluster version 3.0.0 released | AWS ParallelCluster version 3.0.0 released.<br />Support for cluster management via Amazon API Gateway+  Customers can now manage and deploy clusters through HTTP endpoints with Amazon API Gateway. This opens up new possibilities for scripted or event-driven workflows. <br />The AWS ParallelCluster command line interface (CLI) has also been redesigned for compatibility with this API and includes a new JSON output option. This new functionality makes it possible for customers to implement similar building block capabilities using the CLI as well. <br />Improved custom AMI creation+  Customers now have access to a more robust process for creating and managing custom AMIs using EC2 Image Builder. Custom AMIs can now be managed through a separate AWS ParallelCluster configuration file, and can be created using the [pcluster build-image](./pcluster.build-image-v3.html) command in the AWS ParallelCluster command line interface. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster](https://github.com/aws/aws-parallelcluster/blob/v3.0.0/CHANGELOG.md), [aws-parallelcluster-cookbook](https://github.com/aws/aws-parallelcluster-cookbook/blob/v3.0.0/CHANGELOG.md) and [aws-parallelcluster-node](https://github.com/aws/aws-parallelcluster-node/blob/v3.0.0/CHANGELOG.md) packages on GitHub. | September 10, 2021 | 


**PCUI**  

| Change | Description | Date | 
| --- | --- | --- | 
| PCUI version 2026.04.0 released | PCUI version 2026.04.0 released<br />Breaking changes:+  Remove support for AWS ParallelCluster versions <= 3.5.0. <br />Features:+  Add support for managing multiple AWS ParallelCluster versions from a single PCUI deployment. <br />+  Add support for PCUI in `us-gov-east-1` region. <br />Bug fixes:+  Fix an issue to ensure the OS options in the cluster creation wizard reflect the OSes supported by the selected AWS ParallelCluster version. <br />+  Fix an issue to allow the PCUI to offer the full range of selectable instance types in the wizard. <br />+  Improve the CloudFormation error message when the `Version` parameter is not specified. <br />Security:+  Mitigate the risk of path traversal attacks in the PCUI Lambda. <br />+  Fix an OS command injection vulnerability in SSM SendCommand calls on cluster HeadNode. <br />+  Fix a bug causing access tokens to leak into CloudWatch logs. <br />+  Upgrade requests to version 2.33.1 (from 2.32.0). <br />+  Upgrade cryptography to version 46.0.7 (from 44.0.1). <br />+  Upgrade boto3 to version 1.42.91 (from 1.24.30). <br />+  Upgrade urllib3 to version 2.6.3 (from 1.26.20). <br />+  Upgrade ecdsa to version 0.19.2 (from 0.18.0). <br />+  Upgrade certifi to version 2026.2.25 (from 2024.7.4). <br />+  Upgrade flask-cors to version 6.0.2 (from 4.0.2). <br />+  Upgrade python-jose to version 3.5.0 (from 3.4.0). <br />+  Upgrade next to version 15.5.10 (from 14.2.26). <br />+  Upgrade eslint-config-next to version 15.5.10 (from 14.0.4). <br />+  Upgrade @babel/runtime to version 7.27.0 (from 7.23.5). <br />+  Upgrade brace-expansion to version 1.1.12 (from 1.1.11).  | April 30, 2026 | 
| PCUI version 2025.04.0 released | PCUI version 2025.04.0 released<br />Breaking changes:+  Remove default value for the PC version. Now the user must specify the PC version to use. <br />Features:+  Add new stack parameter '`AdditionalPoliciesPCAPI`' to add custom permissions for the ParallelCluster API Lambda role, in addition to the default ones. <br />Bug fixes:+  Fix PCUI deployment in private subnets by making the PCUI template use and return the correct URLs. <br />+  Fix an issue that prevents the loading of 200\+ jobs in the Job status Tab. (See [ https://github.com/aws/aws-parallelcluster-ui/issues/376](https://github.com/aws/aws-parallelcluster-ui/issues/376)). <br />Security:+  Upgrade Python from 3.9 to 3.12. <br />+  Upgrade cross-spawn from 7.0.3 to 7.0.6 to address vulnerability [CVE-2024-21538](https://nvd.nist.gov/vuln/detail/CVE-2024-21538). <br />+  Upgrade requests from 2.31.0 to 2.32.0 to address [CVE-2024-35195](https://nvd.nist.gov/vuln/detail/cve-2024-35195). <br />+  Upgrade urllib3 from 1.26.18 to 1.26.19 to address [CVE-2024-37891](https://nvd.nist.gov/vuln/detail/cve-2024-37891). <br />+  Upgrade cryptography from 42.0.4 to 44.0.1 to address [CWE-1395](https://cwe.mitre.org/data/definitions/1395.html). <br />+  Upgrade certifi from 2023.7.22 to 2024.7.4 to address [CVE-2024-39689](https://nvd.nist.gov/vuln/detail/cve-2024-39689). <br />+  Upgrade jinja2 from 3.1.3 to 3.1.6 to address [CVE-2024-56201](https://nvd.nist.gov/vuln/detail/CVE-2024-56201) and [CVE-2024-56326](https://nvd.nist.gov/vuln/detail/CVE-2024-56326). <br />+  Upgrade serverless\_wsgi.py to version 3.0.5. <br />+  Upgrade Werkzeug from 2.3.8 to version 3.0.6 to address [CVE-2024-34069](https://nvd.nist.gov/vuln/detail/CVE-2024-34069), [CVE-2024-49766](https://nvd.nist.gov/vuln/detail/CVE-2024-49766) and [CVE-2024-49767](https://nvd.nist.gov/vuln/detail/CVE-2024-49767). <br />+  Upgrade Axios from 1.6.7 to version 1.8.2 to address [CVE-2024-39338](https://nvd.nist.gov/vuln/detail/CVE-2024-39338). <br />+  Upgrade Next.js from 14.1.1 to version 14.2.25 to address [CVE-2024-51479](https://nvd.nist.gov/vuln/detail/CVE-2024-51479), [CVE-2024-46982](https://nvd.nist.gov/vuln/detail/CVE-2024-46982) and [CVE-2025-29927](https://nvd.nist.gov/vuln/detail/CVE-2025-29927). <br />+  Upgrade idna from 3.4 to version 3.7 to address [CVE-2024-3651](https://nvd.nist.gov/vuln/detail/CVE-2024-3651). <br />+  Upgrade nanoid from 3.3.7 to version 3.3.8 to address [CVE-2024-55565](https://nvd.nist.gov/vuln/detail/CVE-2024-55565). <br />+  Upgrade python-jose from 3.3.0 to version 3.4.0 to address [CVE-2022-29217](https://nvd.nist.gov/vuln/detail/CVE-2022-29217).  | April 16, 2025 | 
| PCUI version 2024.11.0 released | PCUI version 2024.11.0 released<br />Bug fixes:+  Explicitly set the policy for an ECR private repository to prevent policy removal on a stack update impacting a Lambda function. The policy includes the permissions required by the Lambda function to fetch the code.  | November 22, 2024 | 
| PCUI version 2024.10.0 released | PCUI version 2024.10.0 released<br />Changes:+  Add support for AWS ParallelCluster 3.11.1. <br />+ Add support for On-Demand Capacity Reservations and Capacity Block in the wizard.<br />+ Add g6, m7 and p5 families to the list of supported instance types in the wizard.<br />+ Add new stack optional parameters to configure custom domain for both PCUI and Cognito.<br />Bug fixes:+  Fixes a bug that was breaking the setup of custom domain. <br />Security:+  Upgrade Flask-CORS from 3.0.10 to 4.0.2 to address vulnerability CVE-2024-6221. <br />+  Upgrade lint-staged from 13.0.3 to 15.2.5 to address vulnerability CVE-2024-4068. <br />+ [Full Changelog](https://github.com/aws/aws-parallelcluster-ui/compare/2024.07.1...2024.10.0) | October 22, 2024 | 
| PCUI version 2024.05.0 released | PCUI version 2024.05.0 released.<br />Bug Fixes:+  Fixed a bug in the frontend blocking the UI when the user opens the Job Status panel. <br />+ [Full Changelog](https://github.com/aws/aws-parallelcluster-ui/compare/2024.04.0...2024.05.0) | May 14, 2024 | 
| PCUI version 2024.04.0 released | PCUI version 2024.04.0 released.<br />Features:+  Added support for AWS ParallelCluster version 3.9.1 <br />+ [Full Changelog](https://github.com/aws/aws-parallelcluster-ui/compare/2024.03.0...2024.04.0) | April 17, 2024 | 
| PCUI version 2024.03.0 released | PCUI version 2024.03.0 released.<br />Features:+  Added support for AWS ParallelCluster version 3.9.0 <br />+  Added support for Ubuntu 22.04 and Red Hat Enterprise Linux 9 <br />+  Deprecated Ubuntu 18.04 <br />Bug fixes+  Fixed issue causing some clusters to not appear when using many clusters <br />For details of the changes, see the `CHANGELOG` files for the [`aws-parallelcluster-ui`](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2024.03.0) package on GitHub. | March 12, 2024 | 
| PCUI version 2024.02.0 released | PCUI version 2024.02.0 released<br />Changes:+  Updated the Lambda runtime environment to Python v3.9 <br />For details of the changes, see the `CHANGELOG` files for the [`aws-parallelcluster-ui`](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2024.02.0) package on GitHub. | February 8, 2024 | 
| PCUI version 2023.12.0 released | PCUI version 2023.12.0 released.<br /> <br />Features:+  Added support for PCUI deployment with private networking. <br />+  Added possibility to optionally apply a Permissions Boundary to every IAM role created by the PCUI and PCAPI infrastructures <br />+  Added possibility to optionally apply a prefix to every IAM role and policy created by the PCUI and PCAPI infrastructure.  <br />+  Added support for ParallelCluster version 3.8.0, without feature parity in the wizard. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.12.0) package on GitHub. | December 21, 2023 | 
| PCUI version 2023.10.0 released | PCUI version 2023.10.0 released.<br />Features:+  Added support for ParallelCluster 3.7.2 with feature parity in the wizard limited to FSx File Cache and memory based scheduling compatibility with multiple instance types. <br />Bug fixes:+  Fixed issue causing UI errors when PCUI does not have permissions to interact with Cost Explorer. <br />Improvements+  Improved security by reducing the access token TTL from 10 minutes to 5 minutes. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.10.0) package on GitHub. | October 20, 2023 | 
| PCUI version 2023.06.0 released | PCUI version 2023.06.0 released.<br />Changes:+  Upgraded the default AWS ParallelCluster API version to 3.6.0. <br />Bug fixes:+  Fixed broken deployment for AWS GovCloud (US-West) Region. <br />+  Split panel now correctly loads cluster details after creation has started. <br />Notes:+  The Cost Monitoring feature is not available in AWS GovCloud (US) Regions. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.06.0) package on GitHub. | June 7, 2023 | 
| PCUI version 2023.05.0 released | PCUI version 2023.05.0 released.<br />Enhancements:+  Starting with AWS ParallelCluster version 3.6.0, add support for RHEL 8. <br />+  Add cluster cost monitoring. <br />+  Starting with AWS ParallelCluster version 3.6.0, increase queue and compute resource quotas. <br />Changes:+  Improved the cluster creation wizard user interface. <br />+  Increased the speed of PCUI deployment. <br />+  Improved the interface for adding a new user. <br />+  Queues are in the head node subnet by default. <br />Bug fixes:+  Switch to the correct region after cluster creation completes. <br />+  Fix the loading indicator display in the "Edit cluster" feature. <br />+  Fix cluster creation when the EBS SnapshotId property is removed. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.05.0) package on GitHub. | May 16, 2023 | 
| PCUI version 2023.04.0 released | PCUI version 2023.04.0 released.<br />Enhancements:+  Cluster create wizard re-design. <br />+  Cluster logs page re-design. <br />+  Add custom name setting for shared storage. <br />+  Add multiple storage selection when adding storage to a cluster. <br />+  Add `DeletionPolicy` support for Amazon EFS and FSx for Lustre. <br />+  Add `ImdsSupport` setting in cluster configuration. <br />+  Add support for C7 instance types. <br />+  Added tutorial [Reverting to a previous AWS Systems Manager document version](./tutorials_08_ssm-document-version-rev-v3.html). <br />Changes:+  Cluster configuration YAML up to 1MB in size. <br />+  User isn't logged out due to an authorization with Boto3 IAM temporary credentials. <br />+  Disabled multi-threading options when an HPC instance is selected. <br />+  Removed disable rollback on cluster create page. <br />+  User is prevented from using the PCUI until the required information is provided. <br />+  Up to 10 queues can be added. <br />+  The `SSM-SessionManagerRunShell` document is not overwritten during PCUI installation. <br />Bug fixes:+  Fix broken reset password link. <br />+  Fix broken `delete stack` caused by `EcrPrivateRepository` not being empty <br />+  Fixed initialization issue of the Generate SSH Keys check-box in Multiple user management properties section. <br />+  Fixed crash caused be a job with undefined properties. <br />+  Fixed SCRATCH FSx settings. <br />+  Fixed Start and Stop instances button, still enabled after being clicked once. <br />For details of the changes, see the `CHANGELOG` files for the [aws-parallelcluster-ui](https://github.com/aws/aws-parallelcluster-ui/releases/tag/2023.04.0) package on GitHub. | April 17, 2023 | 


**Terraform**  

| Change | Description | Date | 
| --- | --- | --- | 
| Terraform Provider for AWS ParallelCluster 1.2.0 released | Bug fixes:+ Fixed the `aws-parallelcluster_cluster` data source failing when CloudWatch logging is disabled on the cluster.<br />Changes:+ Upgrade Go minimum version to 1.23.0 (from 1.21).<br />+ Upgrade aws-sdk-go-v2 to v1.36.3 (from v1.32.7).<br />+ Upgrade aws-sdk-go-v2/config to v1.29.13 (from v1.28.7).<br />+ Upgrade aws-sdk-go-v2/service/apigateway to v1.30.1 (from v1.28.2).<br />+ Upgrade aws-sdk-go-v2/service/cloudformation to v1.59.1 (from v1.56.2).<br />+ Upgrade aws-sdk-go-v2/service/sts to v1.33.18 (from v1.33.3).<br />+ Upgrade terraform-plugin-framework to v1.14.1 (from v1.9.0).<br />+ Upgrade terraform-plugin-go to v0.26.0 (from v0.23.0).<br />+ Upgrade terraform-plugin-sdk/v2 to v2.36.1 (from v2.34.0). | August 25, 2026 | 
| Terraform Module for AWS ParallelCluster 1.2.0 released | Changes:+ Allow AWS provider v6.<br />+ Use ParallelCluster API 3.15.1 in all examples. | August 25, 2026 | 
| Terraform Provider for AWS ParallelCluster 1.1.0 released | Bug fixes:+ Fixed an issue that was causing terraform-apply failure when ParallelCluster API 3.11.x is used to deploy clusters with login nodes. | December 6, 2024 | 
| Terraform Module for AWS ParallelCluster 1.1.0 released | Changes:+ Use AWS ParallelCluster Terraform Provider 1.x in all module examples.<br />+ Use ParallelCluster API 3.11.1 in all examples with stack name ParallelClusterAPI.<br />+ Deploy login nodes in all module examples. | December 6, 2024 | 
| Terraform Provider for AWS ParallelCluster 1.0.0 released | Features:+ [Full changelog](https://github.com/aws-tf/terraform-provider-aws-parallelcluster/blob/main/CHANGELOG.md) | June 26, 2024 | 
| Terraform Module for AWS ParallelCluster 1.0.0 released | Features:+ [Full changelog](https://github.com/aws-tf/terraform-aws-parallelcluster/blob/main/CHANGELOG.md) | June 26, 2024 |