Tutorial: Using Amazon EFS file systems with Amazon ECS using the console
Amazon Elastic File System (Amazon EFS) provides simple, scalable file storage for use with your Amazon ECS tasks. With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files. Your applications can have the storage they need, when they need it.
You can use Amazon EFS file systems with Amazon ECS to access file system data across your fleet of Amazon ECS tasks. That way, your tasks have access to the same persistent storage, no matter the infrastructure or container instance on which they land. When you reference your Amazon EFS file system and container mount point in your Amazon ECS task definition, Amazon ECS takes care of mounting the file system in your container. The following sections help you get started using Amazon EFS with Amazon ECS.
This feature is supported by tasks that use both the EC2 and Fargate launch types, however this tutorial will use an Amazon ECS task that uses the EC2 launch type. This tutorial is also meant to be followed step by step, however if you already have some of these resources created on your account then you may be able to skip some steps.
Note
Amazon EFS may not be available in all Regions. For more information about which Regions support Amazon EFS, see Amazon Elastic File System Endpoints and Quotas in the AWS General Reference.
Step 1: Create an Amazon ECS cluster
Use the following steps to create an Amazon ECS cluster.
To create a new cluster (Amazon ECS console)
Before you begin, assign the appropriate IAM permission. For more information, see Cluster examples.
Open the console at https://console.aws.amazon.com/ecs/v2
. -
From the navigation bar, select the Region to use.
-
In the navigation pane, choose Clusters.
-
On the Clusters page, choose Create cluster.
-
Under Cluster configuration, for Cluster name, enter
EFS-tutorial
for the cluster name. -
(Optional) To change the VPC and subnets where your tasks and services launch, under Networking, perform any of the following operations:
-
To remove a subnet, under Subnets, choose X for each subnet that you want to remove.
-
To change to a VPC other than the default VPC, under VPC, choose an existing VPC, and then under Subnets, select each subnet.
-
-
(Optional) To add Amazon EC2 instances to your cluster, expand Infrastructure, and then select Amazon EC2 instances. Next, configure the Auto Scaling group which acts as the capacity provider:
-
To create a Auto Scaling group, from Auto Scaling group (ASG), select Create new group, and then provide the following details about the group:
-
For Operating system/Architecture, choose Amazon Linux 2.
-
For EC2 instance type, choose
t2.micro
.For SSH key pair, choose the pair that proves your identity when you connect to the instance.
-
For Capacity, enter
1
.
-
-
-
Choose Create.
Step 2: Create a security group for the Amazon EFS file system
In this step, you create a security group for your Amazon EFS file system that allows inbound access from your container instances.
Create a security group with the following options:
-
Security group name - a unique name for your security group. For example,
EFS-access-for-sg-
.dc025fa2
-
VPC - the VPC that you identified earlier for your cluster.
-
Inbound rule
-
Type - NFS
-
Source - Custom with the security group ID.
-
For information about how to create a security group, see Create a security group in the Amazon EC2 User Guide for Linux Instances.
Step 3: Create an Amazon EFS file system
In this step, you create an Amazon EFS file system.
To create an Amazon EFS file system for Amazon ECS tasks.
-
Open the Amazon Elastic File System console at https://console.aws.amazon.com/efs/
. -
Choose Create file system.
-
On the Configure network access page, choose the VPC that your container instances are hosted in. By default, each subnet in the specified VPC receives a mount target that uses the default security group for that VPC.
Important
Your Amazon EFS file system, your Amazon ECS cluster, container instances and tasks must be in the same VPC.
-
Under Create mount targets, for Security groups, add the security group that you created in step 2. Choose Next Step.
-
On the Configure file system settings page, configure optional settings and then choose Next Step to proceed.
-
(Optional) Add tags for your file system. For example, you could specify a unique name for the file system by entering that name in the Value column next to the Name key.
-
(Optional) Enable lifecycle management to save money on infrequently accessed storage. For more information, see EFS Lifecycle Management in the Amazon Elastic File System User Guide.
-
Choose a throughput mode for your file system. The Bursting mode is the default, and it is recommended for most file systems.
-
Choose a performance mode for your file system. The General Purpose mode is the default, and it is recommended for most file systems.
-
(Optional) Enable encryption. Select the check box to enable encryption of your Amazon EFS file system at rest.
-
-
On the Configure client access page, choose Next Step.
-
Review your file system options and choose Create File System to complete the process.
-
From the file systems details screen, record the File system ID. In the next step, you will reference this value in your Amazon ECS task definition.
Step 4: Add content to the Amazon EFS file system
In this step, you mount the Amazon EFS file system to an Amazon EC2 instance and add content to it. This is for testing purposes in this tutorial, to illustrate the persistent nature of the data. When using this feature you would normally have your application or another method of writing data to your Amazon EFS file system.
To create an Amazon EC2 instance and mount the Amazon EFS file system
-
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/
. -
Choose Launch Instance.
-
On the Choose an Amazon Machine Image page, select the latest Amazon Linux 2 AMI (HVM) AMI.
-
On the Choose an Instance Type page, keep the default instance type,
t2.micro
and choose Next: Configure Instance Details. -
On the Configure Instance Details page, do the following:
-
For Network, select the VPC that you specified for your Amazon EFS file system and Amazon ECS cluster.
-
For Auto-assign Public IP, choose Enable. Otherwise, your instances do not get public IP addresses or public DNS names.
-
For File systems, select your Amazon EFS file system. You can optionally change the mount location or leave the default value.
-
Under Advanced Details, ensure that the user data script is populated automatically with the Amazon EFS file system mounting steps.
-
-
Advance to step 5 of the instance wizard by choosing Next: Add Storage, Next: Add Tags, and Next: Configure Security Group.
-
On the Configure Security Group page, choose Select an existing security group and select the security group that you created in step 1, and then choose Review and Launch.
-
On the Review Instance Launch page, choose Launch.
-
On the Select an existing key pair or create a new key pair dialog box, select Choose an existing key pair and choose your key pair. Select the acknowledgment check box, and choose Launch Instances.
-
On the Launch Status page, choose View Instances to see the status of your instances. Initially, their status is
pending
. After the status changes torunning
, your instances are ready for use.
Now, you connect to the Amazon EC2 instance and add content to the Amazon EFS file system.
To connect to the Amazon EC2 instance and add content to the Amazon EFS file system
-
SSH to the Amazon EC2 instance you created. For more information, see Connect to Your Linux Instance in the Amazon EC2 User Guide for Linux Instances.
-
From the terminal window for each instance, run the df -T command to verify that the Amazon EFS file system is mounted. In the following output, we have highlighted the Amazon EFS file system mount.
$ df -T Filesystem Type 1K-blocks Used Available Use% Mounted on devtmpfs devtmpfs 485468 0 485468 0% /dev tmpfs tmpfs 503480 0 503480 0% /dev/shm tmpfs tmpfs 503480 424 503056 1% /run tmpfs tmpfs 503480 0 503480 0% /sys/fs/cgroup /dev/xvda1 xfs 8376300 1310952 7065348 16% /
127.0.0.1:/ nfs4 9007199254739968 0 9007199254739968 0% /mnt/efs/fs1
tmpfs tmpfs 100700 0 100700 0% /run/user/1000 -
Navigate to the directory that the Amazon EFS file system is mounted at. In the example above, that is
/mnt/efs/fs1
. -
Create a file named
index.html
with the following content:<html> <body> <h1>It Works!</h1> <p>You are using an Amazon EFS file system for persistent container storage.</p> </body> </html>
Step 5: Create a task definition
The following task definition creates a data volume named efs-html
. The
nginx
container mounts the host data volume at the NGINX root,
/usr/share/nginx/html
.
To create a new task definition using the classic console
Open the console at https://console.aws.amazon.com/ecs/v2
. -
In the navigation pane, choose Task definitions.
-
Choose Create new task definition, Create new task definition with JSON.
-
In the JSON editor box, copy and paste the following JSON text, replacing the
fileSystemId
with the ID of your Amazon EFS file system.{ "containerDefinitions": [ { "memory": 128, "portMappings": [ { "hostPort": 80, "containerPort": 80, "protocol": "tcp" } ], "essential": true, "mountPoints": [ { "containerPath": "
/usr/share/nginx/html
", "sourceVolume": "efs-html
" } ], "name": "nginx", "image": "nginx" } ], "volumes": [ { "name": "efs-html
", "efsVolumeConfiguration": { "fileSystemId": "fs-1324abcd
", "transitEncryption": "ENABLED
" } } ], "family": "efs-tutorial" } -
Choose Save, Create.
Step 6: Run a task and view the results
Now that your Amazon EFS file system is created and there is web content for the NGINX container to serve, you can run a task using the task definition that you created. The NGINX web server serves your simple HTML page. If you update the content in your Amazon EFS file system, those changes are propagated to any containers that have also mounted that file system.
The task runs in the subnet that you defined for the cluster.
To run a task and view the results using the console
Open the console at https://console.aws.amazon.com/ecs/v2
. -
On the Clusters page, select the cluster to run the standalone task in.
Determine the resource from where you launch the service.
To start a service from Steps Clusters
-
On the Clusters page, select the cluster to create the service in.
-
From the Tasks tab, choose Run new task.
Launch type -
On the Task page, choose the task definition.
-
If there is more than one revision, select the revision.
-
Choose Create, Run task.
-
-
(Optional) Choose how your scheduled task is distributed across your cluster infrastructure. Expand Compute configuration, and then do the following:
Distribution method Steps Launch type -
In the Compute options section, select Launch type.
-
For Launch type, choose EC2.
-
-
For Application type, choose Task.
-
For Task definition, choose the
efs-tutorial
task definition that you created earlier .Important
The console validates the selection to ensure that the selected task definition family and revision is compatible with the defined compute configuration.
-
For Desired tasks, enter
1
. -
Choose Create.
-
On the Cluster page, choose Infrastructure.
-
Under Container Instances, choose the container instance to connect to.
-
On the Container Instance page, under Networking, record the Public IP for your instance.
-
Open a browser and enter the public IP address browser, You should see the following message:
It works! You are using an Amazon EFS file system for persistent container storage.
Note
If you do not see the message, make sure that the security group for your container instance allows inbound network traffic on port 80.