Creating users - AWS Transfer Family
Service-managed usersWorking with custom identity providers

Creating users

In this section, you can find information on how to add users using the Transfer Family or a custom identity provider.

If you use a service-managed identity type, you add users to your file transfer protocol enabled server. When you do so, each user name must be unique on your server.

As part of each user's properties, you also store that user's Secure Shell (SSH) public key. Doing so is required for key based authentication, which this getting-started exercise uses. The private key is stored locally on your user's computer. When your user sends an authentication request to your server by using a client, your server first confirms that the user has access to the associated SSH private key. The server then successfully authenticates the user.

In addition, you specify a user's home directory, or landing directory, and assign an AWS Identity and Access Management (IAM) role to the user. Optionally, you can provide a scope-down policy to limit user access only to the home directory of your Amazon S3 bucket.

Service-managed users

You can add either Amazon S3 or Amazon EFS service managed users to your server, depending on the server's Domain setting. For more information, see Creating a server.

Adding Amazon S3 service managed users

To add an Amazon S3 service managed user to your server

  1. Open the Transfer Family console and log in at https://console.aws.amazon.com/transfer/.

  2. On the Servers page, select the check box of the server that you want to add a user to.

  3. Choose Add user.

  4. In the User configuration section, for Username, enter the user name. This user name must be a minimum of 3 and a maximum of 100 characters. You can use the following characters in the user name: a–z, A-Z, 0–9, underscore '_', hyphen '-', period '.', and at sign "@". The user name can't start with a hyphen '-', period '.', or at sign "@".

  5. For Access, choose the IAM role that you previously created that provides access to your Amazon S3 bucket.

    You created this IAM role using the procedure in Create an IAM role and policy. That IAM role includes an IAM policy that provides access to your Amazon S3 bucket. It also includes a trust relationship with the AWS Transfer Family service, defined in another IAM policy.

  6. (Optional) For Policy, choose one of the following:

    • None

    • Existing policy

    • Select a policy from IAM to choose an existing policy. Choose View to see a JSON object containing the details of the policy.

    To learn more about scope-down policy, see Create an IAM role and policy. To learn more about creating a scope-down policy, see Create a scope-down policy.

  7. For Home directory, choose the Amazon S3 bucket to store the data to transfer using AWS Transfer Family. Enter the path to the home directory where your user lands when they log in using their client.

    If you keep this parameter blank, the root directory of your Amazon S3 bucket is used. In this case, make sure that your IAM role provides access to this root directory.

    Note

    We recommend that you choose a directory path that contains the user name of the user, which enables you to effectively use a scope-down policy. The scope-down policy limits user access in the Amazon S3 bucket to that user's home directory.

  8. (Optional) For Restricted, select the check box so that your users can't access anything outside of that folder and can't see the Amazon S3 bucket or folder name.

    Note

    When assigning the user a home directory and restricting the user to that home directory, this should be sufficient enough to lock down the user's access to the designated folder. Use a scope-down policy when you need to apply further controls.

  9. For SSH public key, enter the public SSH key portion of the SSH key pair.

    Your key is validated by the service before you can add your new user.

    Important

    The format of the SSH public key is ssh-rsa <string>. For instructions on how to generate an SSH key pair, see Generate SSH keys.

  10. (Optional) For Key and Value, enter one or more tags as key-value pairs, and choose Add tag.

  11. Choose Add to add your new user to the server that you chose.

    The new user appears in the Users section of the Server details page.

Next steps – For the next step, continue on to Transferring files using a client.

Adding Amazon EFS service managed users

The procedure in this section describes how to add Amazon EFS service managed users to your Transfer Family server.

To add an Amazon EFS service managed user to your server

  1. Open the Transfer Family console (https://console.aws.amazon.com/transfer/) and log in.

  2. On the Servers page, select the Amazon EFS server that you want to add a user to.

  3. Choose Add user to display the Add user page.

  4. In the User configuration section, make the following settings.

    1. For Username, enter the user name. This user name must be a minimum of 3 and a maximum of 100 characters. You can use the following characters in the user name: a–z, A-Z, 0–9, underscore '_', hyphen '-', period '.', and at sign "@". The user name can't start with a hyphen '-', period '.', or at sign "@".

    2. For User ID, enter the user's POSIX user ID. This will be used for all EFS file system operations performed by the user.

    3. For Group ID, enter the user's POSIX group ID. This will be used for all EFS file system operations performed by the user.

    4. (Optional) For Secondary Group IDs, enter one or more additional POSIX group IDs for the user, separated by commas.

    5. For Access, choose the IAM role that:

      • Gives the user the access to only the EFS resources (file systems) that you want them to access.

      • Defines which file system operations that the user can and cannot perform.

      We recommend that you use the role for EFS file system selection with mount access and read/write permissions. For more information, see the blog post New – AWS Transfer Family support for Amazon Elastic File System.

    6. For Home directory, do the following:

      • Choose the EFS file system that you want to store the data to transfer using AWS Transfer Family.

      • Setting the home directory to Restricted does the following:

        • EFS users will not be able to access any files or directories outside of that folder.

        • EFS users will not be able to see the EFS file system name (fs-xxxxxxx).

          Note

          Symlinks will not resolve for EFS users when you select the Restricted option.

      • Optionally, you can enter the path to the home directory where your user lands when they log in using their client.

        If you leave this parameter blank, the root directory of your EFS file system is used. In this case, make sure that your IAM role provides access to this root directory.

  5. For SSH public key, enter the public SSH key portion of the SSH key pair.

    Your key is validated by the service before you can add your new user.

    Important

    The format of the SSH public key is ssh-rsa <string>. For instructions on how to generate an SSH key pair, see Generate SSH keys.

  6. (Optional) Enter any tags for the user. For Key and Value, enter one or more tags as key-value pairs, and choose Add tag.

  7. Choose Add to add your new user to the server that you chose.

    The new user appears in the Users section of the Server details page.

Next steps – For the next step, continue on to Transferring files using a client.

Working with custom identity providers

To integrate your existing identity provider into AWS Transfer Family, provide a RESTful interface with a single Amazon API Gateway method. Transfer Family calls this method to authenticate your users.

This single method authenticates and authorizes your users for access to Amazon S3 or Amazon EFS file systems. After you configure the method, attach it to your server when you create a new server. You can create a new server using the AWS Transfer Family console or CreateServer API operation.

For more information about working with API Gateway, see the API Gateway Developer Guide.

Authenticate using custom identity providers

To create a custom identity provider for Transfer Family, use API Gateway, which provides a highly secure way for you to create and provide APIs. With API Gateway, you can create an HTTPS endpoint so that all incoming API calls are transmitted with greater security. For more details on the API Gateway service, see the API Gateway Developer Guide.

API Gateway offers an authentication method named AWS_IAM, which gives you the same authentication based on AWS Identity and Access Management (IAM) that AWS uses internally. If you enable authentication with AWS_IAM, only callers with explicit permissions to call an API can reach that API's API Gateway method.

To use API Gateway as a custom identity provider for Transfer Family, enable IAM for your API Gateway gateway. As part of this process, you provide an IAM role with permissions for Transfer Family to use your gateway.

To use API Gateway for custom authentication with Transfer Family

  1. Create an AWS CloudFormation stack. To do this:

    1. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation/.

    2. Follow the instructions on deploying an AWS CloudFormation stack from an existing template in Selecting a stack template in the AWS CloudFormation User Guide.

    3. Use one of the following basic templates for creating an AWS Lambda-backed API Gateway for use as a custom identity provider in Transfer Family:

    Deploying one of these stacks is the easiest way to integrate a custom identity provider into the Transfer Family workflow. The stack uses the Lambda function to support a gateway based on API Gateway. You can then use this gateway as a custom identity provider in Transfer Family. By default, the Lambda function authenticates a single user called myuser with a password of MySuperSecretPassword. You can edit these credentials or update the Lambda function code to do something different after deployment.

    Note

    We recommend that you edit the default user and password credentials.

    After the stack has been deployed, you can view details about it on the Outputs tab. These details include the stack Amazon Resource Name (ARN), the ARN of the IAM role that the stack created, and the URL for your new gateway.

    Note

    If you are using the custom identity provider option to enable password–based authentication for your users, and you enable API Gateway's request and response logging, it will log your users' passwords to your Amazon CloudWatch Logs. We don't recommend using this log in your production environment. For more information, see Set up CloudWatch API logging in API Gateway in the API Gateway Developer Guide.

  2. Check the API Gateway configuration for your server. To do this:

    1. Open the API Gateway console at https://console.aws.amazon.com/apigateway/.

    2. Choose the Transfer Custom Identity Provider basic template API that the AWS CloudFormation template generated.

      The following screenshot shows the complete API configuration. In this example, the method is backed by a Lambda function, but many other integration types are also possible.

    3. In the Resources pane, choose GET, and then choose Method Request. The following screenshot shows the correct method configuration that includes protocol, serverId, sourceIp, and username as part of the parameters.

    At this point, your gateway is ready to be deployed.

  3. For Actions, choose Deploy API. For Deployment stage, choose prod, and then choose Deploy.

    After the API is successfully deployed, view its performance in the Stage Editor section, as shown following.

    Note

    Copy the Invoke URL that appears at the top of the screen. You will need it for the next step.

  4. Open the AWS Transfer Family console at https://console.aws.amazon.com/transfer/.

    Choose Create server to open the Create server page. For Choose an identity provider, choose Custom, as shown following.

    For Custom provider, paste the Invoke URL of the API Gateway endpoint that you created in the previous step.

    For Invocation role, choose the IAM role that was created by the AWS CloudFormation template. This role allows Transfer Family to invoke your gateway.

    The invocation role contains the AWS CloudFormation stack name you selected for the stack you created in step 1. It has the following format: CloudFormation-stack-name-TransferIdentityProviderRole-ABC123DEF456GHI.

  5. Fill in the remaining boxes and choose Create server.

Add a web application firewall

AWS WAF is a web application firewall that helps protect web applications and APIs from attacks. You can use it to configure a set of rules known as a web access control list (web ACL) that allow, block, or count web requests based on customizable web security rules and conditions that you define. For more information, see Using AWS WAF to protect your APIs.

To add AWS WAF

  1. Open the API Gateway console at https://console.aws.amazon.com/apigateway/.

  2. In the APIs navigation pane, and then choose your custom identity provider template.

  3. Choose Stages.

  4. In the Stages pane, choose the name of the stage.

  5. In the Stage Editor pane, choose the Settings tab.

  6. Do one of the following:

    • Under Web Application Firewall (WAF), for Web ACL, choose the web ACL that you want to associate with this stage.

    • If the web ACL you need doesn't exist, you will need to create one by doing the following:

      1. Choose Create Web ACL.

      2. On the AWS WAF service homepage, choose Create web ACL.

      3. In Web ACL details, for Name, type the name of the web ACL.

      4. In Rules, choose Add rules, then choose Add my own rules and rule groups.

      5. For Rule type, choose IP set to identify a specific list of IP addresses.

      6. For Rule, enter the name of the rule.

      7. For IP set, choose an existing IP set. To create an IP set, see Creating an IP set.

      8. For IP address to use as the originating address, choose IP address in header.

      9. For Header field name, enter SourceIP.

      10. For Position inside header, choose First IP address.

      11. For Fallback for missing IP address, choose Match or No Match depending on how you want to handle an invalid (or missing) IP address in the header.

      12. For Action, choose the action of the IP set.

      13. For Default web ACL action for requests that don't match any rules, choose Allow or Block and then click Next.

      14. For steps 4 and 5, choose Next.

      15. In Review and create, review your choices, and then choose Create web ACL.

  7. Choose Save Changes.

  8. Choose Resources.

  9. For Actions, choose Deploy API.

Test your configuration

After you create your custom identity provider, you should test your configuration.

Console

To test your configuration by using the AWS Transfer Family console

  1. Open the AWS Transfer Family console at https://console.aws.amazon.com/transfer/.

  2. On the Servers page, choose your new server, choose Actions, and then choose Test.

  3. Enter the text for Username and Password that you set when you deployed the AWS CloudFormation stack. If you kept the default options, the user name is myuser and the password is MySuperSecretPassword.

  4. Choose the Server protocol and enter the IP address for Source IP, if you set them when you deployed the AWS CloudFormation stack.

    If user authentication succeeds, the test returns a StatusCode: 200 HTML response and a JSON object containing the details of the user's roles and permissions, as shown following.

CLI

To test your configuration by using the AWS CLI

  1. Run the test-identity-provider command. 

    aws transfer test-identity-provider --server-id s-1234abcd5678efgh --user-name myuser --user-password MySuperSecretPassword --server-protocol FTP --source-ip 127.0.0.1

  2. Enter the server ID.

  3. Enter the user name and password that you set when you deployed the AWS CloudFormation stack. If you kept the default options, the user name is myuser and the password is MySuperSecretPassword.

  4. Enter the server protocol and source IP, if you set them when you deployed the AWS CloudFormation stack.

    If user authentication succeeds, the command returns a StatusCode: 200 HTML response and a JSON object containing the details of the user's roles and permissions. 

    {
    "Response": "{\"Policy\": \"{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n{\n \"Sid\": \"ReadAndListAllBuckets\",\n 
    \"Effect\": \"Allow\",\n \"Action\": [\n \"s3:ListAllMybuckets\",\n \"s3:GetBucketLocation\",\n \"s3:ListBucket\",\n 
    \"s3:GetObjectVersion\",\n \"s3:GetObjectVersion\"\n],\n \"Resource\":\"*\"\n}\n]\n}\",\"Role\": 
    \"arn:aws:iam::000000000000:role/MyUserS3AccessRole\",\"HomeDirectory\": \"/\"}",
    "StatusCode": 200,
    "Message": "",
    "Url": "https://abcde1234.execute-api.us-east-2.amazonaws.com/prod/servers/s-123a4567bcd891e23/users/myuser/config"
}

Implement your API Gateway method

To create a custom identity provider for Transfer Family, your API Gateway gateway must implement a single method that has a resource path of /servers/serverId/users/username/config. The serverId and username values come from the RESTful resource path. Optionally, you can add the sourceIp and protocol values to the RESTful resource path.

Note

The user name must be a minimum of 3 and a maximum of 100 characters. You can use the following characters in the user name: a–z, A-Z, 0–9, underscore '_', hyphen '-', period '.', and at sign "@". The user name can't start with a hyphen '-', period '.', or at sign "@".

If Transfer Family attempts password authentication for your user, the service supplies a Password: header field. In the absence of a Password: header, Transfer Family attempts public key authentication to authenticate your user.

When using an identity provider to authenticate and authorize end users, in addition to validating their credentials, you can now allow or deny access requests based on the IP addresses of clients used by your end users. This enables you to ensure that data stored in your S3 buckets or Amazon EFS file system can be accessed over the supported protocols only from IP addresses that you have specified as trusted. To enable this, you must include the sourceIp value in the RESTful resource path.

If you have multiple protocols enabled for your server and want to provide access using the same user name over multiple protocols, you can do so as long as the credentials specific to the protocol have been set up in your identity provider. To enable this, you must include the protocol value in the RESTful resource path.

This method should always return HTTP status 200. Any other HTTP status code means an error accessing the API.

Amazon S3 example response

The example response body is a JSON document of the following form for Amazon S3.

{
 "Role": "IAM role with configured S3 permissions",
 "PublicKeys": [
     "ssh-rsa public-key1",
     "ssh-rsa public-key2"
  ],
 "Policy": "STS Assume role scope down policy",
 "HomeDirectory": "/<bucketName>/path/to/home/directory"
}

Amazon EFS example response

The example response body is a JSON document of the following form for Amazon EFS.

{
 "Role": "IAM role with configured EFS permissions",
 "PublicKeys": [
     "ssh-rsa public-key1",
     "ssh-rsa public-key2"
  ],
 "PosixProfile": {
   "Uid": "POSIX user ID",
   "Gid": "POSIX group ID",
   "SecondaryGids": [<Optional list of secondary Group IDs>],
 },
 "HomeDirectory": "/fs-<id>/path/to/home/directory"
}

The Role field shows that successful authentication occurred. When doing password authentication (when you supply a Password: header), you don't need to provide SSH public keys. Also, the Policy and HomeDirectory fields are optional. When no home directory is provided, Transfer Family defaults to the parent S3 bucket or Amazon EFS file system. If a user can't be authenticated, for example if the password is incorrect, your method should return a response without Role set. An example is an empty JSON object.

Include user policies in the Lambda function in JSON format. For more information about configuring user policies in Transfer Family, see Managing your users.

Example Lambda functions for authentication

To implement different authentication strategies, edit the Lambda function that your gateway uses. To help you meet your application's needs, you can find the following example Lambda functions in Python. For more information on Lambda, see the AWS Lambda Developer Guide.

Default Lambda function

The following Lambda function takes your user name, password (in the header field), IAM access role, public SSH key, and protocol. You can use the protocol field to look up your identity provider. Supported protocol values are SFTP, FTPS, and FTP.

Note

If you have multiple protocols enabled for your server and want to provide access using the same user name over multiple protocols, you can do so as long as the credentials specific to the protocol have been set up in your identity provider. For FTP, we recommend maintaining separate credentials from SFTP and FTPS. This is because, unlike SFTP and FTPS, FTP transmits credentials in clear text. By isolating FTP credentials from SFTP or FTPS, if FTP credentials are shared or exposed, your workloads using SFTP or FTPS remain secure.

The Lambda authenticates them, and, if successful, returns the relevant IAM access role and policy. This Lambda function is the same one that is used in the example AWS CloudFormation template.

The HomeDirectoryType parameter is type of landing directory (folder) you want your users' home directory to be when they log into the server. If you set it to PATH, the user will see the absolute Amazon S3 bucket or Amazon EFS paths as is in their file transfer protocol clients. If you set it to LOGICAL, you need to provide mappings in the HomeDirectoryMappings parameter for how you want to make Amazon S3 or EFS paths visible to your users. 

// GetUserConfig Lambda

exports.handler = (event, context, callback) => {
  console.log("Username:", event.username, "ServerId: ", event.serverId);

  var response;
  // Check if the user name presented for authentication is correct. This doesn't check the value of the serverId, only that it is provided.
  // There is also event.protocol (one of "FTP", "FTPS", "SFTP") and event.sourceIp (e.g., "127.0.0.1") to further restrict logins.
  if (event.serverId !== "" && event.username == '${UserName}') {
    response = {
      Role: '${UserRoleArn}', // The user will be authenticated if and only if the Role field is not blank
      Policy: '', // Optional JSON blob to further restrict this user's permissions
      HomeDirectory: '${UserHomeDirectory}' // Not required, defaults to '/'
    };
    
    // Check if password is provided
    if (event.password == "") {
      // If no password provided, return the user's SSH public key
     response['PublicKeys'] = [ "${UserPublicKey1}" ];
    // Check if password is correct
    } else if (event.password !== '${UserPassword}') {
      // Return HTTP status 200 but with no role in the response to indicate authentication failure
      response = {};
    } 
  } else {
    // Return HTTP status 200 but with no role in the response to indicate authentication failure
    response = {};
  }
  callback(null, response);
};

Lambda function for use with AWS Secrets Manager

To use AWS Secrets Manager as your identity provider, you can work with the Lambda function in the sample AWS CloudFormation template. It queries the Secrets Manager service with your credentials and, if successful, returns a designated secret. For more information on Secrets Manager, see the AWS Secrets Manager User Guide.

To download a sample AWS CloudFormation template that uses this Lambda function, go to the Amazon S3 bucket provided by AWS Transfer Family.

Using logical directories to simplify your Transfer Family directory structures

You can simplify your AWS Transfer Family server directory structure using logical directories. Using logical directories, you can construct a virtual directory structure using user-friendly names in which your users navigate when they connect to your Amazon S3 bucket or Amazon EFS file system. When you use logical directories, you can avoid disclosing absolute directory paths, Amazon S3 bucket names, and EFS file system names to your end users.

Use logical directories to set the user’s root directory to a desired location within your storage hierarchy, by performing what is known as a chroot operation. In this mode, users are not able to navigate to a directory outside of the home or root directory that you've configured for them.

For example, although an Amazon S3 user has been scoped down to access only /mybucket/home/${transfer:UserName}, some clients allow users to traverse up a folder to /mybucket/home. In this situation, the user will land back on their intended home directory only after logging out of and back in to the Transfer Family server again. Performing a chroot operation can prevent this from occurring.

You can create your own directory structure across buckets and prefixes. This is useful if you have a workflow that is expecting a specific directory structure that you can't replicate through bucket prefixes. You can also link to multiple non-contiguous locations within Amazon S3, similar to creating a symbolic link in a Linux file system where your directory path references a different location in the file system.

Rules for using logical directories

Before you build your logical directory mappings, you should understand the following rules:

  • When Entry is “/”, you can only have one mapping because overlapping paths are not allowed.

  • Targets can use the ${Transfer:UserName} variable if the bucket or file system path has been parameterized based on their user name.

  • Targets can be paths in different buckets or file systems, but you need to make sure that the mapped IAM role (Role parameter in the response) provides access to those buckets or file systems.

  • You no longer need to specify the HomeDirectory parameter because this value is implied by the Entry Target pairs when using the LOGICAL home directory type option.

Implementing logical directories and chroot

To use logical directories and chroot features, you must do the following:

Turn on logical directories for each user. Do this by returning a new parameter as part of our user configuration. 

"HomeDirectoryType": "LOGICAL"
Chroot

For chroot, create a directory structure that consists of a single Entry and Target pairing for each user. The root folder is the Entry point, and the Target is the location in your bucket or file system to map to. 

[{"Entry": "/", "Target": "/mybucket/jess"}]

You can use an absolute path as in the previous example, or you can use a dynamic substitution for the user name with ${Transfer:UserName}, as in the following example. 

[{"Entry": "/", "Target":
"/mybucket/${Transfer:UserName}"}]

Using this example, the user is locked to their root directory and cannot traverse up higher in the hierarchy.

Virtual directory structure

For a virtual directory structure, create multiple Entry Target pairings, with targets anywhere in your S3 buckets or EFS file systems, including across multiple buckets or file systems, as long as the user’s IAM role mapping has permissions to access them. 

[
{"Entry": "/pics", "Target": "/bucket1/pics"}, 
{"Entry": "/doc", "Target": "/bucket1/anotherpath/docs"},
{"Entry": "/reporting", "Target": "/reportingbucket/Q1"},
{"Entry": "/anotherpath/subpath/financials", "Target": "/reportingbucket/financials"}]

Using this virtual structure example, when the user logs into AWS SFTP, they are in the root directory with sub directories of /pics, /doc, /reporting, and /anotherpath/subpath/financials.

For more information about configuring logical directories and chroot for your users, including an AWS CloudFormation template that you can download and use, see Simplify your AWS SFTP Structure with chroot and logical directories in the AWS Storage Blog.