Moving an environment and resizing or encrypting Amazon EBS volumes - AWS Cloud9

Moving an environment and resizing or encrypting Amazon EBS volumes

You can move an AWS Cloud9 development environment from one Amazon EC2 instance to another. For example, you might want to do the following actions:

  • Transfer an environment from an Amazon EC2 instance that's impaired or performing in unexpected ways compared with a healthy instance.

  • Transfer an environment from an existing instance to one that has the latest system updates.

  • Increase or decrease an instance's compute resources because the environment is overused or underused on the current instance.

You can upgrade from one AWS Cloud9 supported AMI to another by migrating to a new AWS Cloud9 EC2 environment, while keeping the project files. You may want to upgrade to another version of the AMI because:

  • The AMI of the current environment has reached end-of-life and is no longer supported.

  • The package that you need is outdated in the current AMI.

You can also resize the Amazon Elastic Block Store (Amazon EBS) volume that's associated with an Amazon EC2 instance for an environment. For example, you might want to do one or both of the following actions:

  • Increase the size of a volume because you're running out of storage space on the instance.

  • Decrease the size of a volume because you don't want to pay for extra storage space that you aren't using.

Before you move or resize an environment, you can try stopping some running processes in the environment or adding a swap file to the environment. For more information about dealing with low memory or high CPU usage, see Troubleshooting.

Note

This topic only describes moving an environment from one Amazon EC2 instance to another or resizing an Amazon EBS volume. To resize an environment from one of your own servers or to change the storage space for one of your own servers, refer to your server's documentation.

Last, you can encrypt Amazon EBS resources to ensure the security of both data-at-rest and data-in-transit between an instance and its attached EBS storage.

Move an environment

Before you start the move process, note the following conditions:

  • You can't move an environment to an Amazon EC2 instance of the same type. When you move, you must choose a different Amazon EC2 instance type for the new instance.

    Important

    If you move your environment to another Amazon EC2 instance type, that instance type must also be supported by AWS Cloud9 in the current AWS Region. To check the instance types that are available in each Region, go to the Configure settings page that's displayed when creating an EC2 environment with the console. Your choice in the Instance type section is determined by the AWS Region that's selected in the upper right of the console.

  • You must stop the Amazon EC2 instance that's associated with an environment before you can change the instance type. While the instance is stopped, you and any members can't use the environment that's associated with the stopped instance.

  • AWS moves the instance to new hardware, however, the instance's ID doesn't change.

  • If the instance is running in an Amazon VPC and has a public IPv4 address, AWS releases the address and gives the instance a new public IPv4 address. The instance retains its private IPv4 addresses and any Elastic IP addresses or IPv6 addresses.

  • Plan for downtime while your instance is stopped. The process might take several minutes.

To move an environment
  1. (Optional) If the new instance type requires drivers that aren't installed on the existing instance, connect to your instance and install those drivers. For more information, see Compatibility for resizing instances in the Amazon EC2 User Guide for Linux Instances.

  2. Close all web browser tabs that are currently displaying the environment.

    Important

    If you don't close all of the web browser tabs that are currently displaying the environment, AWS Cloud9 might interfere with completing this procedure. Specifically, AWS Cloud9 might try at the wrong time during this procedure to restart the Amazon EC2 instance that's associated with the environment. The instance must stay stopped until the very last step in this procedure.

  3. Sign in to the AWS Management Console, if you're not already signed in, at https://console.aws.amazon.com.

    We recommend you sign in using administrator-level credentials in your AWS account. If you can't do this, check with your AWS account administrator.

  4. Open the Amazon EC2 console. To do this, in the Services list, choose EC2.

  5. In the AWS navigation bar, choose the AWS Region that contains the environment that you want to move (for example, US East (Ohio)).

  6. In the service navigation pane, expand Instances, and then choose Instances.

  7. In the list of instances, choose the one that's associated with the environment that you want to move. For an EC2 environment, the instance name starts with aws-cloud9- followed by the environment name. For example, if the environment is named my-demo-environment, the instance name starts with aws-cloud9-my-demo-environment.

  8. If the Instance State is not Stopped, choose Actions, Instance state, Stop. When prompted, choose Yes, Stop. It can take a few minutes for the instance to stop.

  9. After the Instance State is stopped, with the instance still selected, choose Actions, Instance Settings, Change Instance Type.

  10. In the Change Instance Type dialog box, choose the new Instance Type for the environment to use.

    Note

    If the instance type that you want doesn't appear in the list, it's not compatible with the configuration of the instance. For example, the instance might not be compatible because of the virtualization type.

  11. (Optional) If the instance type that you chose supports EBS–optimization, select EBS-optimized to enable EBS–optimization, or clear EBS-optimized to disable EBS–optimization.

    Note

    If the instance type you chose is EBS–optimized by default, EBS-optimized is selected and you can't clear it.

  12. Choose Apply to accept the new settings.

    Note

    If you didn't choose a different instance type for Instance Type earlier in this procedure, nothing happens after you choose Apply.

  13. Reopen the environment. For more information, see Opening an environment in AWS Cloud9.

For more information about the preceding procedure, see Changing the instance type in the Amazon EC2 User Guide for Linux Instances.

Moving an AWS Cloud9 EC2 environment to a different Amazon Machine Image (AMI)

This topic explains how to migrate an AWS Cloud9 EC2 environment from one Amazon Linux AMI to another AWS Cloud9 supported AMI.

Note

If you want to move your environment to a new instance without updating the OS version, see Move an environment.

You can migrate your data between environments using one of the following procedures:

To move an environment by downloading archive to a local machine
  1. Create a new environment in the same Availability Zone with a different base image:

    1. Complete the steps in the Creating an EC2 Environment section to create a new environment.

      Note

      While choosing the Platform, select the platform that you want to migrate your environment to.

    2. By default, environments are created with 10 GiB volume. If you don't have sufficient space to upload or unpack the archive to the new environment, complete the steps in Resize an Amazon EBS volume that an environment uses procedure to resize Amazon EBS volume size.

  2. Open the environment that you want to migrate in the AWS Cloud9 IDE.

  3. After the AWS Cloud9 IDE loads, select File > Download project from the menu to download the archive with the contents of the environment project directory.

  4. Open AWS Cloud9 IDE in the new environment.

  5. Choose File > Upload local files... to upload the archive.

  6. (Optional) To back up the old .c9 directory to .c9.backup, in the environment terminal, run the following command:

    cp .c9 .c9.backup

    You may need these backup files if you want to restore the configuration files later.

  7. To unpack the archive, run the following command:

    tar xzvf <old_environment_name>.tar.gz -C ~/
  8. To delete the archive from the project directory, run the following command:

    rm <old_environment_name>.tar.gz

    Ensure that the new environment works as expected.

  9. You can now delete the old environment.

To move an environment using Amazon EBS volume

If you are not able to download the archive, or if the resulting archive is too large, you can use the Amazon EBS volume to migrate. Also, this method enables you to copy files that are located outside the ~/environment directory.

  1. Close all AWS Cloud9 IDE tabs that are open in the existing environment.

  2. Complete the following steps to stop the existing instance:

    1. In the AWS Cloud9 console, select the environment to navigate to view its details.

    2. On the Environment details page, under the EC2 instance tab, choose Manage EC2 instance.

    3. In the EC2 console, select the instance to navigate to the instance details.

    4. Ensure that the Instance state is set to Stopped. If not, select Stop instance from the Instance state dropdown list. When prompted, choose Stop. It can take a few minutes for the instance to stop.

  3. Create a new environment in the same Availability Zone with a different base image:

    1. Complete the steps in the Creating an EC2 Environment section to create a new environment.

      Note

      While choosing the Platform, select the platform that you want to migrate your environment to.

    2. By default, environments are created with 10 GiB volume. If you don’t have sufficient space to move files from the source volume to the new environment, complete the steps in Resize an Amazon EBS volume that an environment uses procedure to resize Amazon EBS volume size.

  4. Complete the following steps to detach the volume from the existing instance:

    1. On the Instance summary page, choose the Storage tab and select the volume. The device name of the selected volume must be the same as the one that is specified in the Root device name of the Root device details section.

    2. On the volume details page, choose Actions > Detach volume.

    3. After the volume is successfully detached, choose Actions > Attach volume and then find and select the instance of the new environment from the dropdown list. The name of the Amazon EC2 instance that you select must contain the AWS Cloud9 environment name prefixed with aws-cloud9.

  5. Open AWS Cloud9 IDE in the new environment.

  6. After the environment loads, to identify the device of the newly attached volume, run the following command in the terminal:

    lsblk

    In the following sample output, partition nvme0n1 of root device nvme0n1p1 is already mounted, hence the nvme1n1p1 partition must also be mounted. The full path for its device is /dev/nvme1n1p1:

    Admin:~/environment $ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS nvme0n1 259:0 0 10G 0 disk ├─nvme0n1p1 259:2 0 10G 0 part / ├─nvme0n1p127 259:3 0 1M 0 part └─nvme0n1p128 259:4 0 10M 0 part /boot/efi nvme1n1 259:1 0 10G 0 disk ├─nvme1n1p1 259:5 0 10G 0 part └─nvme1n1p128 259:6 0 1M 0 part
    Note

    The output varies when you run this command in your terminal.

  7. Complete the following steps in the environment terminal to mount the existing volume:

    1. To create a temporary directory where the volume’s partition will be mounted, run the following command:

      MOUNT_POINT=$(mktemp -d)
    2. Based on the lsblk command's sample output, specify the following path of the device to be mounted:

      MOUNT_DEVICE=/dev/nvme1n1p1
      Note

      The output varies when you run this command in your terminal.

    3. To mount the existing volume, run the following command:

      sudo mount $MOUNT_DEVICE $MOUNT_POINT
    4. Complete the following steps to verify if the existing volume is correctly mounted:

      1. To ensure that the volume is included in the output, run the following command:

        df -h
      2. To verify contents of the volume, run the following command:

        ls $MOUNT_POINT/home/ec2-user/environment/
  8. (Optional) To back up the old .c9 directory to .c9.backup, in the environment terminal, run the following command:

    cp .c9 .c9.backup

    You may need these backup files if you want to restore the configuration files later.

  9. To copy the old environment from the existing volume, run the following command:

    cp -R $MOUNT_POINT/home/ec2-user/environment ~
    Note

    If required, you can also copy files or directories outside of the environment directory using the preceding command.

    Ensure that the new environment works as expected.

  10. To unmount the previous device, run one of the two following commands:

    sudo umount $MOUNT_DEVICE
    sudo umount $MOUNT_POINT
  11. Choose Detach volume from the Actions dropdown list to detach the volume that you attached in Step 3.

  12. You can now delete the old environment and its volume.

    Note

    Since the volume is no longer attached to the environment’s Amazon EC2 instance, you’ll need to remove it manually. You can do this by choosing Delete on the Volume details page.

Resize an Amazon EBS volume that an environment uses

  1. Open the environment that's associated with the Amazon EC2 instance for the Amazon EBS volume that you want to resize.

  2. In the AWS Cloud9 IDE for the environment, create a file with the following contents, and then save the file with the extension .sh (for example, resize.sh).

    Note

    This script works for Amazon EBS volumes that are connected to EC2 instances that run AL2023, Amazon Linux 2, Amazon Linux, or Ubuntu Server and is configured to use IMDSv2.

    The script also resizes Amazon EBS volumes exposed as NVMe block devices on Nitro-based instances. For a list of instances based on the Nitro system, see Nitro-based instances in the Amazon EC2 User Guide for Linux Instances.

    #!/bin/bash # Specify the desired volume size in GiB as a command line argument. If not specified, default to 20 GiB. SIZE=${1:-20} # Get the ID of the environment host Amazon EC2 instance. TOKEN=$(curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 60") INSTANCEID=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/instance-id 2> /dev/null) REGION=$(curl -s -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/placement/region 2> /dev/null) # Get the ID of the Amazon EBS volume associated with the instance. VOLUMEID=$(aws ec2 describe-instances \ --instance-id $INSTANCEID \ --query "Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.VolumeId" \ --output text \ --region $REGION) # Resize the EBS volume. aws ec2 modify-volume --volume-id $VOLUMEID --size $SIZE # Wait for the resize to finish. while [ \ "$(aws ec2 describe-volumes-modifications \ --volume-id $VOLUMEID \ --filters Name=modification-state,Values="optimizing","completed" \ --query "length(VolumesModifications)"\ --output text)" != "1" ]; do sleep 1 done # Check if we're on an NVMe filesystem if [[ -e "/dev/xvda" && $(readlink -f /dev/xvda) = "/dev/xvda" ]] then # Rewrite the partition table so that the partition takes up all the space that it can. sudo growpart /dev/xvda 1 # Expand the size of the file system. # Check if we're on AL2 or AL2023 STR=$(cat /etc/os-release) SUBAL2="VERSION_ID=\"2\"" SUBAL2023="VERSION_ID=\"2023\"" if [[ "$STR" == *"$SUBAL2"* || "$STR" == *"$SUBAL2023"* ]] then sudo xfs_growfs -d / else sudo resize2fs /dev/xvda1 fi else # Rewrite the partition table so that the partition takes up all the space that it can. sudo growpart /dev/nvme0n1 1 # Expand the size of the file system. # Check if we're on AL2 or AL2023 STR=$(cat /etc/os-release) SUBAL2="VERSION_ID=\"2\"" SUBAL2023="VERSION_ID=\"2023\"" if [[ "$STR" == *"$SUBAL2"* || "$STR" == *"$SUBAL2023"* ]] then sudo xfs_growfs -d / else sudo resize2fs /dev/nvme0n1p1 fi fi
  3. From a terminal session in the IDE, switch to the directory that contains the resize.sh file. Then run either of the following commands, replacing 20 with the size in GiB that you want to resize the Amazon EBS volume to:

    • bash resize.sh 20
    • chmod +x resize.sh ./resize.sh 20

Encrypt Amazon EBS volumes that AWS Cloud9 uses

Amazon EBS encryption encrypts the following data:

  • Data at rest in the volume

  • All data that moves between the volume and the instance

  • All snapshots that are created from the volume

  • All volumes that are created from those snapshots

You have two encryption options for Amazon EBS volumes that are used by AWS Cloud9 EC2 development environments:

  • Encryption by default – You can configure your AWS account to enforce the encryption of the new EBS volumes and snapshot copies that you create. Encryption by default is enabled at the level of an AWS Region. So, you can't enable it for individual volumes or snapshots in that Region. In addition, Amazon EBS encrypts the volume that's created when you launch an instance. So, you must enable this setting before you create an EC2 environment. For more information, see Encryption by default in the Amazon EC2 User Guide for Linux Instances.

  • Encryption of an existing Amazon EBS volume used by an EC2 environment – You can encrypt specific Amazon EBS volumes that are already created for EC2 instances. This option involves using the AWS Key Management Service (AWS KMS) to manage access to the encrypted volumes. For the relevant procedure, see Encrypt an existing Amazon EBS volume that AWS Cloud9 uses.

Important

If your AWS Cloud9 IDE uses Amazon EBS volumes that are encrypted by default, the AWS Identity and Access Management service-linked role for AWS Cloud9 requires access to the AWS KMS key for these EBS volumes. If access isn't provided, the AWS Cloud9 IDE might fail to launch and debugging might be difficult.

To provide access, add the service-linked role for AWS Cloud9, AWSServiceRoleForAWSCloud9, to the KMS key that's used by your Amazon EBS volumes. For more information about this task, see Create an AWS Cloud9 IDE that uses Amazon EBS volumes with default encryption in AWS Prescriptive Guidance Patterns.

Encrypt an existing Amazon EBS volume that AWS Cloud9 uses

Encrypting an existing Amazon EBS volume involves using AWS KMS to create a KMS key. After you create a snapshot of the volume to replace, you use the KMS key to encrypt a copy of the snapshot.

Next, you create an encrypted volume with that snapshot. Then, you replace the unencrypted volume by detaching it from the EC2 instance and attaching the encrypted volume.

Finally, you must update the key policy for the customer managed key to enable access for the AWS Cloud9 service role.

Note

The following procedure focuses on using a customer managed key to encrypt a volume. You can also use an AWS managed key for an AWS service in your account. The alias for Amazon EBS is aws/ebs. If you choose this default option for encryption, skip step 1 where you create a customer managed key. Also, skip step 8 where you update the key policy. This is because you can't change the key policy for an AWS managed key.

To encrypt an existing Amazon EBS volume
  1. In the AWS KMS console, create a symmetric KMS key. For more information, see Creating symmetric KMS key in the AWS Key Management Service Developer Guide.

  2. In the Amazon EC2 console, stop the Amazon EBS-backed instance used by the environment. You can stop the instance using the console or the command line.

  3. In the navigation pane of the Amazon EC2 console, choose Snapshots to create a snapshot of the existing volume that you want to encrypt.

  4. In the navigation pane of the Amazon EC2 console, choose Snapshots to copy the snapshot. In the Copy snapshot dialog box, do the following to enable encryption:

    • Choose Encrypt this snapshot.

    • For Master Key, select the KMS key that you created earlier. (If you're using an AWS managed key, keep the (default) aws/ebs setting.)

  5. Create a new volume from the encrypted snapshot.

    Note

    New Amazon EBS volumes that are created from encrypted snapshots are automatically encrypted.

  6. Detach the old Amazon EBS volume from the Amazon EC2 instance.

  7. Attach the new encrypted volume to the Amazon EC2 instance.

  8. Update the key policy for the KMS key using the AWS Management Console default view, AWS Management Console policy view, or AWS KMS API. Add the following key policy statements to allow the AWS Cloud9 service, AWSServiceRoleForAWSCloud9, to access the KMS key.

    Note

    If you're using an AWS managed key, skip this step.

    { "Sid": "Allow use of the key", "Effect": "Allow", "Principal": { "AWS": "arn:{Partition}:iam::{AccountId}:role/aws-service-role/cloud9.amazonaws.com/AWSServiceRoleForAWSCloud9" }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ], "Resource": "*" }, { "Sid": "Allow attachment of persistent resources", "Effect": "Allow", "Principal": { "AWS": "arn:{Partition}:iam::{AccountId}:role/aws-service-role/cloud9.amazonaws.com/AWSServiceRoleForAWSCloud9" }, "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ], "Resource": "*", "Condition": { "Bool": { "kms:GrantIsForAWSResource": "true" } } }
  9. Restart the Amazon EC2 instance. For more information about restarting an Amazon EC2 instance, see Stop and start your instance.