Configure FSx for OpenZFS as principal storage for diskless hosts
This page applies to Self-deployed environments that run VCF 9.x on diskless hosts. For how EVS-deployed environments (VCF 5.2.x) differ, see Differences between deployment modes.
Diskless hosts need external storage for the VMware Cloud Foundation (VCF) management domain. You size and create an Amazon FSx for OpenZFS file system and two Network File System (NFS) volumes, and you set the security group rules that they need. You can grow the capacity later. You mount the volumes on the first host and reference the principal volume in the VCF Installer specification, as described in Step 4a (diskless): Prepare diskless hosts and the VCF Installer and Step 4b (diskless): Create the specification and deploy. For an overview of the storage options, see External storage options for diskless hosts.
Before you begin
Make sure that you meet the following prerequisites.
-
You created an Amazon Elastic VMware Service environment in Self-deployed mode in your VPC. For more information, see Step 1: Create the environment.
-
You know the CIDR block of the host management (
vmkManagement) VLAN subnet. You use it in the NFS export list and in the security group. -
You know the ID of the service access subnet that Amazon EVS uses.
-
You or your storage administrator have permissions to create and manage FSx for OpenZFS file systems in your VPC. For more information, see Identity and access management for Amazon FSx for OpenZFS in the Amazon FSx for OpenZFS User Guide.
Size the file system
The VCF Installer checks the capacity of the principal datastore during validation. The datastore size that the ESX host reports depends on the file system and on the volume quota. A volume with no quota appears on the ESX host with the capacity of the file system. Provision enough capacity for the Installer requirement plus the amount that the Installer advises to stay under 80 percent utilization.
For the two-host management domain specification in Step 4b (diskless): Create the specification and deploy, the VCF Installer reported the following values.
| Installer check | Value |
|---|---|
|
Required capacity for the management domain |
5,398 GB |
|
Recommended capacity to stay under 80 percent utilization |
6,748 GB |
The VCF Installer reports capacity in GB. Amazon FSx capacity and the recommendation that follows are in GiB.
For a new deployment of this size, we recommend that you provision at least 7,168 GiB. A file system with 7,168 GiB of SSD storage capacity and no volume quota appears on the ESX host as a 7,168 GiB datastore. At this size, both Installer capacity checks pass and the Installer shows no capacity advisory. If you set a volume quota, keep it above 6,748 GB.
After the two-host management domain is deployed, the principal datastore holds about 284 GiB. The datastores are thin provisioned, so the file system leaves a large amount of free space for workloads. Because both volumes share the file system, the second volume can show a slightly smaller size than the principal volume. Use the size of the principal datastore for capacity planning.
You can increase the SSD storage capacity of a file system later, but you can’t decrease it. Plan the capacity before you deploy VCF. For more information, see Increase the capacity.
For prices, see the Amazon FSx for OpenZFS pricing page
Create the file system
Create a file system in the service access subnet of your environment. You can use the Amazon FSx console or the AWS CLI.
Before you create the file system, create a security group in your VPC with the rules in Security group rules.
Create the file system with the option to copy tags to volumes turned off.
If this option is on and the file system has a Name tag, creating a volume with a Name tag that has a different value fails with a BadRequest error.
To create the file system in the Amazon FSx console or with the AWS CLI, use the following steps.
Example
Keep the no_root_squash option, because ESX mounts NFS datastores as the root user.
The example turns off automatic backups.
Choose the backup and maintenance settings that match your requirements.
For more information, see Creating FSx for OpenZFS file systems in the Amazon FSx for OpenZFS User Guide.
A Single-AZ (HA) generation 2 file system with 7,168 GiB of SSD storage and 640 MBps of throughput becomes available in 10 to 25 minutes. Wait until the file system is available before you create the volumes. The endpoint IP address appears when the file system is available.
If you created the file system with the option to copy tags to volumes turned on, turn it off with the following command.
The change causes no interruption.
Replace <file-system-id> with the ID of your file system, and replace <region-code> with the Region where you created the environment.
aws fsx update-file-system \ --file-system-id <file-system-id> \ --open-zfs-configuration CopyTagsToVolumes=false \ --region <region-code>
An Amazon EVS environment runs in one Availability Zone. For more information, see VPC and subnets. A Multi-AZ file system keeps the storage available if an Availability Zone fails, but it doesn’t keep your Amazon EVS environment running. Choose the deployment type for your availability and cost requirements. For more information, see Availability and durability in the Amazon FSx for OpenZFS User Guide.
Security group rules
The FSx for OpenZFS file system doesn’t start outbound connections. The NFS datastores mount through the host management VMkernel adapter, so allow the following inbound traffic from the CIDR block of the host management VLAN subnet.
| Purpose | Protocol | Ports |
|---|---|---|
|
NFS and related services |
TCP and UDP |
111, 2049, and 20001 to 20003 |
|
Connectivity checks |
Internet Control Message Protocol (ICMP) |
All |
For more information, see File System Access Control with Amazon VPC in the Amazon FSx for OpenZFS User Guide.
Create the volumes
Create two child volumes of the root volume.
-
vcf-principal -
The principal datastore for the VCF management domain. The VCF Installer deploys the management appliances onto this volume.
-
installer-staging -
The temporary datastore for the VCF Installer appliance. You mount it on the first host before you deploy the VCF Installer. During deployment, the VCF Installer moves its appliance to the principal datastore.
Create each volume with the following settings.
- Parent volume
-
Use the root volume of the file system.
- NFS exports
-
Use the host management VLAN subnet CIDR block with the options
rw,crossmnt,no_root_squash. - Storage capacity quota
-
Use none (
-1). The volume is then limited by the quota of the root volume, which equals the file system capacity. The describe output of a volume without a quota shows no quota value. A smaller quota limits the datastore size that ESX reports, so keep any quota above 6,748 GB. - Record size
-
The examples on this page use 16 KiB for
vcf-principaland 128 KiB forinstaller-staging. - Tags
-
If you add a
Nametag, make sure that the option to copy tags to volumes is turned off on the file system.
To create the volumes in the Amazon FSx console or with the AWS CLI, use the following steps.
Example
You have created the two volumes.
Get the endpoint IP address
You use the endpoint IP address of the file system as the NFS server when you mount the volumes on the host and in the VCF Installer specification. To get the endpoint IP address in the Amazon FSx console or with the AWS CLI, use the following steps.
Example
The NFS export path of a volume is /fsx/<volume-name>.
For example, the path of the principal volume is /fsx/vcf-principal.
Increase the capacity
If you need more capacity before you deploy VCF, complete the following steps in order. After you deploy VCF, the principal datastore holds your management appliances, and you can’t unmount a datastore while virtual machines use it.
-
Increase the SSD storage capacity of the file system. Each increase must be at least 10 percent of the current capacity. After a change, you must wait six hours before you can change the capacity again. Replace
<file-system-id>with the ID of your file system, and replace the example capacity with your own value. Replace<region-code>with the Region where you created the environment.aws fsx update-file-system \ --file-system-id <file-system-id> \ --storage-capacity 9216 \ --region <region-code> -
If you set a storage capacity quota on the volume, raise the quota. Each volume has its own quota, and a volume doesn’t use the new capacity until you raise it. Replace
<volume-id>with the ID of the volume.aws fsx update-volume \ --volume-id <volume-id> \ --open-zfs-configuration StorageCapacityQuotaGiB=9216 \ --region <region-code> -
Unmount the NFS datastore from the host and mount it again, so that ESX reads the new size. A storage rescan doesn’t update the reported size.
To delete the file system when you no longer need it, see Clean up a Self-deployed environment.
Next step: Mount the volumes on the first host, as described in Prepare the first host.