Troubleshooting AWS Transfer Family

Use the following information to help you diagnose and fix common issues that you might encounter when working with AWS Transfer Family.

For issues with IAM in Transfer Family, see Troubleshooting AWS Transfer Family identity and access.

Troubleshoot service-managed users

This section describes possible solutions for the following issues

Topics

• Troubleshoot Amazon EFS service-managed users
• Troubleshoot public key body too long
• Troubleshoot failed to add SSH public key

Troubleshoot Amazon EFS service-managed users

Description

You run the sftp command and the prompt doesn't appear, and instead you see the following message:

Couldn't canonicalize: Permission denied
           Need cwd

Cause

Your AWS Identity and Access Management (IAM) user's role does not have permission to access Amazon Elastic File System (Amazon EFS).

Solution

Increase the policy permissions for your user's role. You can add an AWS managed policy, such as AmazonElasticFileSystemClientFullAccess.

Troubleshoot public key body too long

Description

When you try to create a service-managed user, you receive the following error:

Failed to create user (1 validation error detected:
'sshPublicKeyBody' failed to satisfy constraint: Member must have length less than or equal to 2048)

Cause

You might be entering a PGP key for the public key body, and AWS Transfer Family does not support PGP keys for service-managed users.

Solution

If the PGP key is RSA-based, you can convert it to PEM format. For example, Ubuntu provides a conversion tool here: https://manpages.ubuntu.com/manpages/xenial/man1/openpgp2ssh.1.html

Troubleshoot failed to add SSH public key

Description

When you try to add a public key for a service-managed user, you receive the following error:

Failed to add SSH public key (Unsupported or invalid SSH public key format)

Cause

You might be attempting to import an SSH2-formatted public key, and AWS Transfer Family does not support SSH2-formatted public keys for service-managed users.

Solution

You need to convert the key into OpenSSH format. This process is described in Convert an SSH2 public key to PEM format.

Troubleshoot Amazon API Gateway issues

This section describes possible solutions for the following issues:

Topics

• Too many authentication failures
• Connection closed

Too many authentication failures

Description

When you try to connect to your server using Secure Shell (SSH) File Transfer Protocol (SFTP), you get the following error:

Received disconnect from 3.15.127.197 port 22:2: Too many authentication failures
Authentication failed.
Couldn't read packet: Connection reset by peer

Cause

You might have entered an incorrect password for your user. Try again to enter the correct password.

If the password is correct, the issue might be caused by a role Amazon Resource Name (ARN) that is not valid. To confirm that this is the issue, test the identity provider for your server. If you see a response similar to the following, the role ARN is a placeholder only, as indicated by the role ID value of all zeros:

{
    "Response": "{\"Role\": \"arn:aws:iam::000000000000:role/MyUserS3AccessRole\",\"HomeDirectory\": \"/\"}",
    "StatusCode": 200,
    "Message": "",
    "Url": "https://api-gateway-ID.execute-api.us-east-1.amazonaws.com/prod/servers/transfer-server-ID/users/myuser/config"
}       

Solution

Replace the placeholder role ARN with an actual role that has permission to access the server.

To update the role

1. Open the AWS CloudFormation console at https://console.aws.amazon.com/cloudformation.

2. In the left navigation pane, choose Stacks.

3. In the Stacks list, choose your stack, and then choose the Parameters tab.

4. Choose Update. On the Update stack page, choose Use current template, and then choose Next.

5. Replace UserRoleArn with a role ARN that has sufficient permissions for accessing your Transfer Family server.

   Note

   To grant the necessary permissions, you can add the AmazonAPIGatewayAdministrator and the AmazonS3FullAccess managed policies to your role.

6. Choose Next, and then choose Next again. On the Review stack page, select I acknowledge that AWS CloudFormation might create IAM resources, and then choose Update stack.

Connection closed

Description

When you try to connect to your server using Secure Shell (SSH) File Transfer Protocol (SFTP), you get the following error:

Connection closed

Cause

One possible cause for this issue is that your Amazon CloudWatch logging role does not have a trust relationship with Transfer Family.

Solution

Make sure that the logging role for the server has a trust relationship with Transfer Family. For more information, see To establish a trust relationship.

Troubleshoot policies for encrypted Amazon S3 buckets

Description

You have an encrypted Amazon S3 bucket that you are using as storage for your Transfer Family server. If you try to upload a file to the server, you receive the error Couldn't close file: Permission denied.

And if you view the server logs, you see the following errors:

ERROR Message="Access denied" Operation=CLOSE Path=/bucket/user/test.txt BytesIn=13
ERROR Message="Access denied"

Cause

The policy for your IAM user does not have permission to access the encrypted bucket.

Solution

You must specify additional permissions in your policy to grant the required AWS Key Management Service (AWS KMS) permissions. For details, see Data encryption in Amazon S3.

Troubleshoot authentication issues

This section describes possible solutions for the following issues:

Topics

• Authentication failures—SSH/SFTP
• Managed AD mismatched realms issue
• Miscellaneous authentication issues

Authentication failures—SSH/SFTP

Description

When you try to connect to your server using Secure Shell (SSH) File Transfer Protocol (SFTP), you receive a message similar to the following:

Received disconnect from 3.130.115.105 port 22:2: Too many authentication failures
  Authentication failed.

Note

If you are using an API Gateway and receive this error, see Too many authentication failures.

Cause

You have not added an RSA key pair for your user, so you must authenticate using a password instead.

Solution

When you run the sftp command, specify the -o PubkeyAuthentication=no option. This option forces the system to request your password. For example:

sftp -o PubkeyAuthentication=no [e-mail]

Managed AD mismatched realms issue

Description

A user's realm and their group realm must match. They must both be in the default realm, or they must both be in the trusted realm.

Cause

If a user and their group do not match, the user cannot be authenticated by Transfer Family. If you test the identity provider for the user, you receive the error No associated access found for user's groups.

Solution

Reference a group in the user's realm that matches the group realm (either default or trusted).

Miscellaneous authentication issues

Description

You receive an authentication error and none of the other troubleshooting works

Cause

You might have specified a target for a logical directory that contains a leading or trailing slash (/).

Solution

Update your logical directory target, to make sure it begins with a slash, and does not contain a trailing slash. For example, /DOC-EXAMPLE-BUCKET/images is acceptable, but DOC-EXAMPLE-BUCKET/images and /DOC-EXAMPLE-BUCKET/images/ are not.

Troubleshoot managed workflows issues

This section describes possible solutions for the following issues:

Topics

• Troubleshoot workflow-related errors using Amazon CloudWatch
• Troubleshoot workflow copy errors

Troubleshoot workflow-related errors using Amazon CloudWatch

Description

If you are having issues with your workflows, you can use Amazon CloudWatch to investigate the cause.

Cause

There can be several causes. Use Amazon CloudWatch Logs to investigate.

Solution

Transfer Family emits workflow execution status into CloudWatch Logs. The following types of workflow errors can appear in CloudWatch Logs:

• "type": "StepErrored"

• "type": "ExecutionErrored"

• "type": "ExecutionThrottled"

• "Service failure on starting workflow"

You can filter your workflow's execution logs using different filter and pattern syntax. For example, you can create a log filter in your CloudWatch logs to capture workflow execution logs that contain the ExecutionErrored message. For details, see Real-time processing of log data with subscriptions and Filter and pattern syntax in the Amazon CloudWatch Logs User Guide.

StepErrored

2021-10-29T12:57:26.272-05:00
            {"type":"StepErrored","details":{"errorType":"BAD_REQUEST","errorMessage":"Cannot
            tag Efs file","stepType":"TAG","stepName":"successful_tag_step"},
            "workflowId":"w-abcdef01234567890","executionId":"1234abcd-56ef-78gh-90ij-1234klmno567",
            "transferDetails":{"serverId":"s-1234567890abcdef0","username":"lhr","sessionId":"1234567890abcdef0"}

Here, StepErrored indicates that a step within the workflow has generated an error. In a single workflow, you can have multiple steps configured. This error tells you in which step the error occurred and provides an error message. In this particular example, the step was configured to tag a file; however, tagging a file in an Amazon EFS file system is not supported, so the step generated an error.

ExecutionErrored

2021-10-29T12:57:26.618-05:00
            {"type":"ExecutionErrored","details":{},"workflowId":"w-w-abcdef01234567890",
            "executionId":"1234abcd-56ef-78gh-90ij-1234klmno567","transferDetails":{"serverId":"s-1234567890abcdef0",
            "username":"lhr","sessionId":"1234567890abcdef0"}}

When a workflow is not able to execute any step, it generates an ExecutionErrored message. For example, if you have configured a single step in a given workflow, and if the step is not able to execute, the overall workflow fails.

Executionthrottled

Execution is throttled if a workflow is getting triggered at a rate that is faster than the system can support. This log message indicates that you must slow down your execution rate for workflows. If you are not able to scale down your workflow-execution rate, contact AWS Support at Contact AWS.

Service failure on starting workflow

Anytime you remove a workflow from a server and replace it with a new one, or update server configuration (which impacts a workflow's execution role), you must wait approximately 10 minutes before executing the new workflow. The Transfer Family server caches the workflow details, and it takes 10 minutes for the server to refresh its cache.

Additionally, you must log out of any active SFTP sessions, and then log back in after the 10-minute waiting period to see the changes.

Troubleshoot workflow copy errors

Description

If you're executing a workflow that contains a step to copy the uploaded file, you could encounter the following error:

{
  "type": "StepErrored", "details": {
    "errorType": "BAD_REQUEST", "errorMessage": "Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request;
    Request ID: request-ID; S3 Extended Request ID: request-ID Proxy: null)", "stepType": "COPY", "stepName": "copy-step-name" },
  "workflowId": "workflow-ID",
  "executionId": "execution-ID",
  "transferDetails": {
     "serverId": "server-ID",
     "username": "user-name",
     "sessionId": "session-ID"
  }
}

Cause

The source file is in an Amazon S3 bucket that is in a different AWS Region than the destination bucket.

Solution

If you're executing a workflow that includes a copy step, make sure that the source and destination buckets are in the same AWS Region.

Troubleshoot workflow decryption issues

Description

Your decrypt workflow fails, and the log message resembles the following:

{
   "type": "StepErrored",
   "details": {
      "errorType": "BAD_REQUEST",
      "errorMessage": "File encryption algorithm not supported with FIPS mode enabled.",
      "stepType": "DECRYPT",
      "stepName": "step-name"
   },
   "workflowId": "workflow-ID",
   "executionId": "execution-ID",
   "transferDetails": {
      "serverId": "server-ID",
      "username": "user-name",
      "sessionId": "session-ID"
   }
}

Cause

Your Transfer Family server has FIPS mode enabled and an associated Decrypt workflow step. When encrypting the files before uploading to your Transfer Family server, the encryption client might generate encrypted files that use non-FIPS approved symmetric encryption algorithms. In such a scenario, the workflow is unable to decrypt files. In the following example, GnuPG version 2.4.0 is using OCB (a non-FIPS block cipher mode) to encrypt files: this causes the workflow to fail.

Solution

You must edit the GPG key that you used to encrypt your files, and then re-encrypt them. The following procedure describes the steps you must take.

To edit your PGP keys

1. Identify the key that you must edit by running gpg ‐‐list-keys

   This returns a list of keys. Each key has details similar to the following:

   pub   ed25519 2022-07-07 [SC]
         wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
   uid           [ultimate] Mary Major <[e-mail]>
   sub   cv25519 2022-07-07 [E]

2. Identify the key that you want to edit. In the example shown in the previous step, the ID is wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.

3. Run gpg ‐‐edit-key wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.

   The system responds with details about the GnuPG program and the specified key.

4. At the gpg> prompt, enter showpref. The following details are returned:

   [ultimate] (1). Mary Major <[e-mail]>
      Cipher: AES256, AES192, AES, 3DES
      AEAD: OCB
      Digest: SHA512, SHA384, SHA256, SHA224, SHA1
      Compression: ZLIB, BZIP2, ZIP, Uncompressed
      Features: MDC, AEAD, Keyserver no-modify

   Note that the preferred algorithms that are stored on the key are listed.

5. We want to edit the key to retain all algorithms except for OCB. Run the setpref command, specifying all the algorithms to retain:

   gpg> setpref AES256, AES192,AES,3DES,SHA512, SHA384, SHA256, SHA224, SHA1,ZLIB, BZIP2, ZIP, Uncompressed

   This returns the following details:

   Set preference list to:
        Cipher: AES256, AES192, AES, 3DES
        AEAD: 
        Digest: SHA512, SHA384, SHA256, SHA224, SHA1
        Compression: ZLIB, BZIP2, ZIP, Uncompressed
        Features: MDC, Keyserver no-modify
   Really update the preferences? (y/N)

6. Enter y to update, then enter your password when prompted to confirm the change.

7. Save the changes.

   gpg> save

Before re-running your decrypt workflow, you must re-encrypt your files, using the edited key.

Troubleshoot Amazon EFS issues

This section describes possible solutions for the following issues.

Topics

• Troubleshoot missing POSIX profile
• Troubleshoot logical directories with Amazon EFS

Troubleshoot missing POSIX profile

Description

If you're using Amazon EFS storage for your server and you're using a custom identity provider, you must provide your AWS Lambda function with a POSIX profile.

Cause

One possible cause is that the templates that we provide for creating an AWS Lambda-backed Amazon API Gateway method do not currently contain POSIX information.

If you did provide POSIX information, the format that you used for providing the POSIX information might not be getting parsed correctly by Transfer Family.

Solution

Make sure that you are providing a JSON element to Transfer Family for the PosixProfile parameter.

For example, if you're using Python, you could add the following line where you parse the PosixProfile parameter:

if PosixProfile: 
        response_data["PosixProfile"] = json.loads(PosixProfile)

Or, in JavaScript, you could add the following line, where the uid-value and gid-value are integers, 0 or greater, that represent the User ID (UID) and Group ID (GID) respectively:

PosixProfile: {"Uid": uid-value, "Gid": gid-value},

These code examples send the PosixProfile parameter to Transfer Family as a JSON object, rather than as a string.

Also, within AWS Secrets Manager, you must store the PosixProfile parameter as follows. Replace your-uid and your-gid with your actual values for the GID and UID.

{"Uid": your-uid, "Gid": your-gid, "SecondaryGids": []}

Troubleshoot logical directories with Amazon EFS

Description

If the user's home directory does not exist, and they run an ls command, the system responds as follows:

sftp> ls
remote readdir ("/"): No such file or directory

Cause

If your Transfer Family server uses Amazon EFS, the home directory for the user must be created with read and write access before the user can work in their logical home directory. The user cannot create this directory themselves, as they would lack permissions for mkdir on their logical home directory.

Solution

A user with administrative access to the parent directory needs to create the user's logical home directory.

Troubleshoot testing your identity provider

Description

If you test your identity provider using the console or the TestIdentityProvider API call, the Response field is empty. For example:

{
    "Response": "{}",
    "StatusCode": 200,
    "Message": ""
}

Cause

The most likely cause is that the authentication failed because of an incorrect user name or password.

Solution

Make sure that you are using the correct credentials for your user, and make updates to the username or password, if necessary.

Troubleshoot adding trusted host keys for your SFTP connector

Description

When you are creating or editing an SFTP connector, and you are adding a trusted host key, you receive the following error: Failed to edit connector details (Invalid host key format.)

Cause

If you paste in a correct public key, the issue might be that you included the comment portion of the key. AWS Transfer Family does not currently accept the comment portion of the key.

Solution

Delete the comment portion of the key, when you paste it into the text field. For example, assume your key looks similar to the following:

ssh-rsa AAAA...== [e-mail]

Remove the text that follows the == characters and only paste in the portion of the key up to and including the ==.

ssh-rsa AAAA...==

Troubleshoot file upload issues

This section describes possible solutions for the following issues:

Topics

• Troubleshoot Amazon S3 file upload errors
• Troubleshoot unreadable file names

Troubleshoot Amazon S3 file upload errors

Description

When you are attempting to upload a file to Amazon S3 storage using Transfer Family, you receive the following error message: AWS Transfer does not support random access writes to S3 objects.

Cause

When you're using Amazon S3 for your server's storage, Transfer Family does not support multiple connections for a single transfer.

Solution

If your Transfer Family server is using Amazon S3 for its storage, disable any options for your client software that mention using multiple connections for a single transfer.

Troubleshoot unreadable file names

Description

You see corrupted file names in some of your uploaded files. Users sometimes encounter problems with FTP and SFTP transfers that garble certain characters in file names, such as umlauts, accented letters, or certain scripts, such as Chinese or Arabic.

Cause

Although the FTP and SFTP protocols can allow for character encoding of files names to be negotiated by clients, Amazon S3 and Amazon EFS do not. Instead, they require UTF-8 character encoding. As a result, certain characters are not rendered correctly.

Solution

To solve this problem, review your client application for file name character encoding and make sure it is set to UTF-8.

Troubleshoot ResourceNotFound exception

Description

You receive an error where the resource cannot be found. For example, if you run UpdateServer, you might get the following error:

An error occurred (ResourceNotFoundException) when calling the UpdateServer operation: Unknown server

Cause

There are several reasons for receiving a ResourceNotFoundException message. In most cases, the resource that you specified in your API command does not exist. If you did specify an existing resource, then the most probable cause is that your default region is different than the region for your resource. For example, if your default region is us-east-1, and your Transfer Family server is in us-east-2, you will receive an Unknown resource exception.

For details about setting a default region, see Quick configuration with aws configure.

Solution

Add a region parameter to your API command to explicitly specify where to find a particular resource.

aws transfer -describe-server --server-id server-id --region us-east-2

Troubleshoot SFTP connector issues

Description

You receive an error where the key exchange negotiation fails. For example:

Key exchange negotiation failed due to incompatible host key algorithms. 
Client offered: [ecdsa-sha2-nistp256, ecdsa-sha2-nistp384,
ecdsa-sha2-nistp521, rsa-sha2-512, rsa-sha2-256] Server offered: [ssh-rsa]

Cause

This error is because there's no overlap between the host key algorithms supported by the server and those supported by the connector.

Solution

Ensure that the remote server supports at least one of the Client host key algorithms listed in the error message. For the list of supported algorithms, see SFTP connector algorithms.

Troubleshoot AS2 issues

Error messages and troubleshooting tips for Applicability Statement 2 (AS2)-enabled servers are described here: AS2 error codes.