How Amazon FSx for NetApp ONTAP works - FSx for ONTAP

How Amazon FSx for NetApp ONTAP works

This topics describes the major features of FSx for ONTAP and how they work, along with important implementation details.

FSx for ONTAP file systems

An FSx for ONTAP file system is composed of the following primary resources:

  • Storage virtual machines (SVMs)

  • Volumes

A file system can have one or more SVMs, and an SVM can have one or more volumes. The following image shows the structure of FSx for ONTAP file systems, and the relationship of the three primary resources. The FSx for ONTAP file system on the left is the simplest file system with one SVM and one volume. The file system on the right has multiple SVMs, with some SVMs having multiple volumes. File systems and SVMs each have multiple endpoints for management, with SVMs also having endpoints for data access.


    Structure of FSx for ONTAP file systems.

FSx for ONTAP file systems

A file system is the primary resource in Amazon FSx (analgous to an ONTAP cluster on-premises). You specify the SSD storage capacity and throughput capacity for your file system, and choose an Amazon Virtual Private Cloud (VPC) in which your file system is created. Each file system has a management endpoint that you can optionally use to manage your data using the ONTAP CLI or ONTAP REST API. Data stored in a file system can be automatically tiered to lower-cost storage if data-tiering is enabled.

You define the following properties when creating an FSx for ONTAP file system:

  • Storage capacity – The amount of SSD storage, up to 192 TiB.

  • SSD IOPS – By default, each gigabyte of SSD storage includes 3 SSD IOPS. You can optionally provision additional SSD IOPS as needed.

  • Throughput capacity – The sustained speed at which the file server can serve data.

  • Networking – The VPC, subnets, routes tables, IP address range for the management and data access endpoints your file system creates.

  • Encryption – The AWS Key Management Service (KMS) key used to encrypt the file system data at rest.

  • Administrative access – You can specify the password for the "fsxadmin" user, which you can use to administer the file system using the NetApp ONTAP CLI and REST API.

For more information, see Managing FSx for ONTAP file systems.

File system endpoints

You can manage FSx for ONTAP file systems using the NetApp ONTAP CLI, REST API, and you can set up SnapMirror relationships between an Amazon FSx file system and another ONTAP deployment (including another Amazon FSx file system). Each Amazon FSx for NetApp ONTAP file system has the following file system endpoints that provide access to NetApp applications:

  • Management – Use this endpoint to access the ONTAP CLI over SSH, or to use the ONTAP REST API with your file system.

  • Intercluster – Use this endpoint when setting up replication using NetApp SnapMirror.

For more information, see Managing FSx for ONTAP resources using NetApp applications and Scheduled replication using NetApp SnapMirror.

Storage virtual machines (SVM)

A storage virtual machine (SVM) is an isolated file server with its own administrative credentials and endpoints for administering and accessing data. When you access data on Amazon FSx for NetApp ONTAP, your clients and workstations access the endpoint for the SVM in which the data is stored. Amazon FSx automatically creates a default SVM on your file system for you when you create a file system using the AWS Management Console.

Each SVM is a virtual resource, meaning that the SVMs in your file system share your file system’s storage and throughput capacity. Because each SVM is an isolated file server, if you have multiple users or groups who need access to administer data on Amazon FSx, you can create a separate SVM for each user or group so that they can independently administer their data. You can also configure quality of service (QoS) policies within your file system to limit the amount of throughput and IOPS that individual workloads can drive, ensuring that individual workloads don’t interfere with the other users and groups on the same file system. You can create additional SVMs on your file system at any time using the AWS Management Console, AWS CLI, or Amazon FSx API and SDKs.

You define the following properties when creating an SVM:

  • The file system to which it belongs

  • Active Directory configuration – You can optionally join your SVM to a self-managed Microsoft Active Directory for authentication and access control of Windows and Mac clients.

  • Root volume security style – Sets the root volume security style (Unix, NTFS, or Mixed) to align with the type of clients you're using to access your data within the SVM.

  • The SvmAdminPassword, which is the password for the SVM's vsadmin user.

SVM endpoints

Each SVM has four endpoints that are used to access data or to manage the SVM using the NetApp ONTAP CLI, REST API, listed as follows:

  • Nfs – for connecting using the NFS protocol

  • Smb – for connecting using the SMB protocol (if your SVM is joined to an Active Directory)

  • Iscsi – for connecting using the iSCSI protocol.

  • Management – for managing SVMs using the NetApp ONTAP CLI, NetApp ONTAP API, or NetApp CloudManager.

For more information, see Managing FSx for ONTAP storage virtual machines.

The following table lists the maximum number of SVMs you can create for a file system depending on the amount of throughput capacity provisioned and the protocols used to access volumes.

Amount of throughput capacity (MB/s) Maximum number of SVMs per file system

512

14

1024

14

2048

24

Volumes

Volumes are isolated data containers in which your files, directories, or iSCSI LUNs are stored. Volumes are thin provisioned, meaning that they only consume storage capacity for the data stored in them. Each volume is created within one of the SVMs in your file system.

You can create volumes using the AWS Management Console, AWS CLI, the Amazon FSx API, or using NetApp Cloud Manager. You can also use your file system’s or SVM’s administrative endpoint to create, update, and delete volumes using the ONTAP CLI or ONTAP REST API.

When you create a volume, you define the following properties:

  • The name of the volume.

  • The size of the volume.

  • The junction path, which is the location in the SVM's namespace where the volume is mounted.

  • You can enable storage efficiency to use compression, deduplication, and compaction to reduce the amount of storage your data consumes.

  • Set the volume security style (Unix, NTFS, or Mixed) to match the majority of clients that you expect to be accessing the volume.

  • You can enable automatic data tiering and set which tiering policy to use. For more information, see Data tiering.

You can create up to 500 volumes per file system. For more information, see Managing FSx for ONTAP volumes.

Data tiering

When you create an Amazon FSx for NetApp ONTAP file system, you provision a level of SSD storage capacity. As you write data to your file system, your less frequently-accessed data is automatically transitioned to the capacity pool tier, a lower-cost storage tier that automatically grows and shrinks with the amount of data tiered to it. As a result, you only need to provision as much SSD storage as needed for the active portion of your data set, with the rest of your data stored in lower-cost capacity pool storage. Amazon FSx automatically and intelligently transitions data between storage tiers based on your access patterns, allowing you to achieve SSD levels of performance for your workload while only paying for SSD storage for a small fraction of your data.

Each volume in your Amazon FSx for NetApp ONTAP file system has a tiering policy associated with it, which determines how the data within that volume is transitioned to and from capacity pool storage. You can choose from one of four tiering policies:

  • Auto moves cold user data blocks in the active file system and in Snapshot copies to the storage pool tier.

    If read by random reads, the cold data blocks in the capacity tier become hot and move to the primary storage tier. If read by sequential reads, such as those associated with index and antivirus scans, the cold data blocks stay cold and do not move to the primary storage tier.

  • Snapshot-only moves user data blocks of the volume Snapshot copies that are not associated with the active file system to the storage pool tier.

    If read, cold data blocks on the capacity tier become hot and are moved to the primary storage tier.

  • All moves moves all user data blocks in the the active file system and in Snapshot copies to the storage pool tier.

    If read, cold data blocks on the cloud tier stay cold and are not written back to the performance tier.

  • None keeps data of a volume in the primary storage tier, preventing it from being moved to the storage pool tier.

For the Auto and Snapshot-only tiering policies, you can also specify a minimum cooling period, which sets the time that user data in a volume must remain inactive for the data to be considered cold and moved to the capacity pool tier. The minimum cooling period, which applies to both Snapshot and active file system data, ranges from 2 to 183 days. For Auto the default cooling period is 31 days, for Snapshot-only the default cooling period is 2 days.

When you create a new volume, data tiering is not enabled by default. For information about enabling data tiering on a volume, see Managing FSx for ONTAP volumes.

Using Microsoft Active Directory

You can use your existing Microsoft Active Directory (AD) for user authentication and authorization. To do so, you need to join any SVMs hosting volumes that will be accessed by Windows or macOS SMB clients to your AD. You specify your AD's configuration when you create an SVM. The following are the AD properties you need to set when joining an SVM to your AD:

  • NetBiosName – name of the computer object that's created in your Active Directory for your SVM.

  • DnsIps – Up to 3 IP addresses for your DNS servers.

  • DomainName – The fully qualified domain name of the self-managed AD directory.

  • FileSystemAdministratorsGroup – The name of the domain group whose members have administrative privileges on your SVM.

  • Credentials – The user name and password for a service account in your self-managed AD domain that Amazon FSx uses to join to your AD domain.

Keeping your AD configuration updated in Amazon FSx

To help ensure continued, uninterrupted availability of your Amazon FSx file system, update your SVM's self-managed Active Directory (AD) configuration any time that you make changes to your self-managed AD setup.

For example, suppose that your AD uses a time-based password reset policy. In this case, as soon as the password is reset, make sure to update the service account password with Amazon FSx. To do this, use the Amazon FSx console, API, or AWS CLI. Similarly, if the DNS server IP addresses change for your AD domain, as soon as the change occurs update the DNS server IP addresses with Amazon FSx. Again, do this using the Amazon FSx console, API, or CLI.

For more information, see Managing FSx for ONTAP storage virtual machines.

Accessing data stored on Amazon FSx for NetApp ONTAP file systems

You can access the data on your FSx for ONTAP file systems using Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), Amazon WorkSpaces, Amazon AppStream 2.0, VMware Cloud on AWS, and on-premises clients.

You can access an ONTAP volume from multiple Linux, Windows, or macOS clients simultaneously over the NFS (v3, v4, v4.1, v4.2) and SMB protocols. You can also use the iSCSI protocol to access iSCSI LUNs. For more information, see Accessing data: supported clients and environments.

Amazon FSx for NetApp ONTAP file systems can be accessed from on-premises using AWS VPN or AWS Direct Connect with AWS Transit Gateway (TGW). You can also use TGW or VPC Peering to access clusters from another VPC (including a VPC in another AWS Region). For more information, see Supported access environments.

How to work with Amazon FSx for NetApp ONTAP

There are several ways that you can interact with FSx for ONTAP. You can manage your Amazon FSx for NetApp ONTAP file systems using the following AWS and NetApp management applications and tools:

  • AWS management tools:

    • The AWS Management Console

    • The AWS Command Line Interface (AWS CLI)

    • The Amazon FSx APIs and SDKs

  • NetApp management tools:

    • NetApp CloudManager

    • The NetApp ONTAP CLI

    • The NetApp ONTAP REST API

AWS Management Console

The AWS Management Console is a simple web-based user interface. You can manage your Amazon FSx file systems from the console with no programming required. To access the Amazon FSx console, sign in to the AWS Management Console, and then open the Amazon FSx console at https://console.aws.amazon.com/fsx/.

Amazon FSx command line interface (CLI)

You can use the AWS CLI to access the Amazon FSx API interactively. To install the AWS CLI, see Installing, updating and uninstalling the AWS CLI. To begin using the AWS CLI for Amazon FSx, see AWS Command Line Interface reference for Amazon FSx.

Amazon FSx application programming interface (API)

If you're a developer, you use the Amazon FSx API actions and data types to programmatically configure and manage Amazon FSx and its resources. You can also use the API in one of the language-specific AWS software development kits (SDKs). For more information, see the Amazon FSx API reference.

For application development, we recommend that you use one of the AWS SDKs. The AWS SDKs handle low-level details such as authentication, retry logic, and error handling, so that you can focus on your application logic. The AWS SDKs are available for a wide variety of languages. For more information, see Tools to Build on AWS.

AWS also provides libraries, sample code, tutorials, and other resources to help you get started more easily. For more information, see the AWS Developer Center.

NetApp Cloud Manager

NetApp Cloud Manager provides a centralized user interface to manage, monitor, and automate ONTAP deployments in AWS and on premises. For more information, see Managing FSx for ONTAP resources using NetApp applications.

NetApp ONTAP CLI

You can use the NetApp ONTAP CLI to manage your Amazon FSx for NetApp ONTAP file systems. The ONTAP CLI provides a command-based view of the ONTAP management interface. You enter commands at the cluster management endpoint as the fsxadmin user, or at the SVM management endpoint as the vsadmin user. For more information, see Managing FSx for ONTAP resources using NetApp applications.

NetApp ONTAP REST API

You can use the NetApp ONTAP REST API to manage your Amazon FSx for NetApp ONTAP file systems. You can access the ONTAP REST API using your file system's management endpoint, or using the management endpoint associated with any SVM. For more information, see Managing FSx for ONTAP resources using NetApp applications.