Monitoring Transfer Family usage - AWS Transfer Family
Enabling CloudTrail loggingCreating Amazon CloudWatch alarmsLogging S3 API calls to S3 access logsLogging activity with CloudWatchExamples to limit confused deputy problemExample CloudWatch log entriesUsing CloudWatch metricsUser notifications

Monitoring Transfer Family usage

You can monitor activity in your server using Amazon CloudWatch and AWS CloudTrail. For further analysis, you can also record server activity as readable, near real-time metrics. To view metrics that are specific to Transfer Family servers that have the AS2 protocol enabled, see Monitoring AS2 usage.

Enable AWS CloudTrail logging

You can monitor AWS Transfer Family API calls using AWS CloudTrail. By monitoring API calls, you can get useful security and operational information. For more information about how to work with CloudTrail and AWS Transfer Family, see Logging and monitoring in AWS Transfer Family.

If you have Amazon S3 object level logging enabled, RoleSessionName is contained in the Requester field as [AWS:Role Unique Identifier]/username.sessionid@server-id. For more information about AWS Identity and Access Management (IAM) role unique identifiers, see Unique identifiers in the AWS Identity and Access Management User Guide.

Important

The maximum length of the RoleSessionName is 64 characters. If the RoleSessionName is longer, the server-id gets truncated.

Creating Amazon CloudWatch alarms

The following example shows how to create Amazon CloudWatch alarms using the AWS Transfer Family metric, FilesIn.

CDK
new cloudwatch.Metric({
  namespace: "AWS/Transfer",
  metricName: "FilesIn",
  dimensionsMap: { ServerId: "s-00000000000000000" },
  statistic: "Average",
  period: cdk.Duration.minutes(1),
}).createAlarm(this, "AWS/Transfer FilesIn", {
  threshold: 1000,
  evaluationPeriods: 10,
  datapointsToAlarm: 5,
  comparisonOperator: cloudwatch.ComparisonOperator.GREATER_THAN_OR_EQUAL_TO_THRESHOLD,
});
AWS CloudFormation
Type: AWS::CloudWatch::Alarm
Properties:
  Namespace: AWS/Transfer
  MetricName: FilesIn
  Dimensions:
    - Name: ServerId
      Value: s-00000000000000000
  Statistic: Average
  Period: 60
  Threshold: 1000
  EvaluationPeriods: 10
  DatapointsToAlarm: 5
  ComparisonOperator: GreaterThanOrEqualToThreshold

Logging Amazon S3 API calls to S3 access logs

If you are using Amazon S3 access logs to identify S3 requests made on behalf of your file transfer users, RoleSessionName is used to display which IAM role was assumed to service the file transfers. It also displays additional information such as the user name, session id, and server-id used for the transfers. The format is [AWS:Role Unique Identifier]/username.sessionid@server-id and is contained in the Requester field. For example, the following are the contents for a sample Requester field from an S3 access log for a file that was copied to the S3 bucket.

arn:aws:sts::AWS-Account-ID:assumed-role/IamRoleName/username.sessionid@server-id

In the Requester field above, it shows the IAM Role called IamRoleName. For more information about IAM role unique identifiers, see Unique identifiers in the AWS Identity and Access Management User Guide.

Log activity with CloudWatch

To set access, you create a resource-based IAM policy and an IAM role that provides that access information.

To enable Amazon CloudWatch logging, you start by creating an IAM policy that enables CloudWatch logging. You then create an IAM role and attach the policy to it. You can do this when you are creating a server or by editing an existing server. For more information about CloudWatch, see What is Amazon CloudWatch? and What is Amazon CloudWatch logs? in the Amazon CloudWatch User Guide.

Use the following example IAM policies to allow CloudWatch logging.

Use a logging role
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "logs:DescribeLogStreams",
                "logs:CreateLogGroup",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:*:*:log-group:/aws/transfer/*"
        }
    ]
}
Use structured logging
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
            "logs:CreateLogDelivery",
                "logs:GetLogDelivery",
                "logs:UpdateLogDelivery",
                "logs:DeleteLogDelivery",
                "logs:ListLogDeliveries",
                "logs:PutResourcePolicy",
                "logs:DescribeResourcePolicies",
                "logs:DescribeLogGroups"
            ],
            "Resource": "arn:aws:logs:region-id:AWS account:log-group:/aws/transfer/*"
        }
    ]
}

In the preceding example policy, for the Resource, replace the region-id and AWS account with your values. For example, "Resource": "arn:aws::logs:us-east-1:111122223333:log-group:/aws/transfer/*"

You then create a role and attach the CloudWatch Logs policy that you created.

To create an IAM role and attach a policy

  1. In the navigation pane, choose Roles, and then choose Create role.

    On the Create role page, make sure that AWS service is chosen.

  2. Choose Transfer from the service list, and then choose Next: Permissions. This establishes a trust relationship between AWS Transfer Family and the IAM role. Additionally, add aws:SourceAccount and aws:SourceArn condition keys to protect yourself against the confused deputy problem. See the following documentation for more details:

  3. In the Attach permissions policies section, locate and choose the CloudWatch Logs policy that you just created, and choose Next: Tags.

  4. (Optional) Enter a key and value for a tag, and choose Next: Review.

  5. On the Review page, enter a name and description for your new role, and then choose Create role.

  6. To view the logs, choose the Server ID to open the server configuration page, and choose View logs. You are redirected to the CloudWatch console where you can see your log streams.

On the CloudWatch page for your server, you can see records of user authentication (success and failure), data uploads (PUT operations), and data downloads (GET operations).

Examples to limit confused deputy problem

The confused deputy problem is a security issue where an entity that doesn't have permission to perform an action can coerce a more-privileged entity to perform the action. In AWS, cross-service impersonation can result in the confused deputy problem. For more details, see Cross-service confused deputy prevention.

Note

In the following examples, replace each user input placeholder with your own information.

The following example logging/invocation policy allows any server in the account to assume the role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowAllServers",
            "Effect": "Allow",
            "Principal": {
                "Service": "transfer.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "account-id"
                },
                "ArnLike": {
                    "aws:SourceArn": "arn:aws:transfer:region:account-id:server/*"
                }
            }
        }
    ]
}

The following example logging/invocation policy allows a specific server to assume the role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSpecificServer",
            "Effect": "Allow",
            "Principal": {
                "Service": "transfer.amazonaws.com"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "aws:SourceAccount": "account-id"
                },
                "ArnEquals": {
                    "aws:SourceArn": "arn:aws:transfer:region:account-id:server/server-id"
                }
            }
        }
    ]
}

Example CloudWatch log entries

The following table contains example log entries for various Transfer Family actions.

Action Corresponding logs within Amazon CloudWatch Logs
Logins/Logouts

user.914984e553bcddb6 CONNECTED SourceIP=1.22.111.222 User=lhr HomeDir=LOGICAL Client=SSH-2.0-OpenSSH_7.4 Role=arn:aws::iam::123456789012:role/sftp-s3-access

user.914984e553bcddb6 DISCONNECTED
Uploads

lhr.33a8fb495ffb383b OPEN Path=/bucket/user/123.jpg Mode=CREATE|TRUNCATE|WRITE

lhr.33a8fb495ffb383b CLOSE Path=/bucket/user/123.jpg BytesIn=3618546
Downloads

lhr.33a8fb495ffb383b OPEN Path=/bucket/user/123.jpg Mode=READ

llhr.33a8fb495ffb383b CLOSE Path=/bucket/user/123.jpg BytesOut=3618546
Deletes

lhr.33a8fb495ffb383b DELETE Path=/bucket/user/123.jpg
Renames

lhr.33a8fb495ffb383b RENAME Path=/bucket/user/lambo.png NewPath=/bucket/user/ferrari.png

Symlinks

lhr.eb49cf7b8651e6d5 CREATE_SYMLINK LinkPath=/fs-12345678/lhr/pqr.jpg TargetPath=abc.jpg

Authentication failures

ERRORS AUTH_FAILURE Method=publickey User=lhr Message="RSA SHA256:Lfz3R2nmLY4raK+b7Rb1rSvUIbAE+a+Hxg0c7l1JIZ0" SourceIP=3.8.172.211

Workflows

{"type":"ExecutionStarted","details":{"input":{"initialFileLocation":{"backingStore":"EFS","filesystemId":"fs-12345678","path":"/lhr/regex.py"}}},"workflowId":"w-1111aaaa2222bbbb3","executionId":"1234abcd-1234-efgh-5678-ijklmnopqr90","transferDetails":{"serverId":"s-zzzz1111aaaa22223","username":"lhr","sessionId":"1234567890abcdef0"}}

{"type":"StepStarted","details":{"input":{"fileLocation":{"backingStore":"EFS","filesystemId":"fs-12345678","path":"/lhr/regex.py"}},"stepType":"CUSTOM","stepName":"efs-s3_copy_2"},"workflowId":"w-9283e49d33297c3f7","executionId":"1234abcd-1234-efgh-5678-ijklmnopqr90","transferDetails":{"serverId":"s-18ca49dce5d842e0b","username":"lhr","sessionId":"1234567890abcdef0"}}
Custom step workflow

{"type":"CustomStepInvoked","details":{"output":{"token":"MzM4Mjg5YWUtYTEzMy00YjIzLWI3OGMtYzU4OGI2ZjQyMzE5"},"stepType":"CUSTOM","stepName":"efs-s3_copy_2"},"workflowId":"w-9283e49d33297c3f7","executionId":"1234abcd-1234-efgh-5678-ijklmnopqr90","transferDetails":{"serverId":"s-zzzz1111aaaa22223","username":"lhr","sessionId":"1234567890abcdef0"}}
COPY/TAG/DELETE/DECRYPT workflow

{"type":"StepStarted","details":{"input":{"fileLocation":{"backingStore":"EFS","filesystemId":"fs-12345678","path":"/lhr/regex.py"}},"stepType":"TAG","stepName":"successful_tag_step"},"workflowId":"w-1111aaaa2222bbbb3","executionId":"81234abcd-1234-efgh-5678-ijklmnopqr90","transferDetails":{"serverId":"s-1234abcd5678efghi","username":"lhr","sessionId":"1234567890abcdef0"}}
Sample workflow error log

{"type":"StepErrored","details":{"errorType":"BAD_REQUEST","errorMessage":"Cannot tag Efs file","stepType":"TAG","stepName":"successful_tag_step"},"workflowId":"w-1234abcd5678efghi","executionId":"81234abcd-1234-efgh-5678-ijklmnopqr90","transferDetails":{"serverId":"s-1234abcd5678efghi","username":"lhr","sessionId":"1234567890abcdef0"}}

To view your Transfer Family server logs

  1. Navigate to the details page for a server.

  2. Choose View logs. This opens Amazon CloudWatch.

  3. The log group for your selected server is displayed.

    The log group details page. In addition to the log group, it presents a list the recent log streams.

  4. You can select a log stream to display details and individual entries for the stream.

    • If there is a listing for ERRORS, you can choose it to view details for the latest errors for the server.

      The logging for ERRORS example page. This example shows details for several authentication failures.

    • Choose any other entry to see an example log stream.

      An example logging screen, showing a session where a user connects, and then later disconnects, from the SFTP server.

    • If your server has a managed workflow associated with it, you can view logs for the workflow runs.

      Note

      The format for the log stream for the workflow is username.workflowId.uniqueStreamSuffix. For example, decrypt-user.w-a1111222233334444.aaaa1111bbbb2222 could be the name of a log stream for user decrypt-user and workflow w-a1111222233334444.

      An example logging screen, showing entries for one run through a workflow, with the StepStarted log entry expanded.
Note

For any expanded log entry, you can copy the entry to the clipboard by choosing Copy. For more details about CloudWatch logs, see Viewing log data.

Using CloudWatch metrics for Transfer Family

Note

You can also get metrics for Transfer Family from within the Transfer Family console itself. For details, see Monitoring usage in the console

You can get information about your server using CloudWatch metrics. A metric represents a time-ordered set of data points that are published to CloudWatch. When using metrics, you must specify the Transfer Family namespace, metric name, and dimension. For more information about metrics, see Metrics in the Amazon CloudWatch User Guide.

The following table describes the CloudWatch metrics for Transfer Family.

Namespace Metric Description

AWS/Transfer

BytesIn

The total number of bytes transferred into the server.

Units: Count

Period: 5 minutes

BytesOut

The total number of bytes transferred out of the server.

Unit: Count

Period: 5 minutes

FilesIn

The total number of files transferred into the server.

For servers using the AS2 protocol, this metric represents the number of messages received.

Units: Count

Period: 5 minutes

FilesOut

The total number of files transferred out of the server.

Units: Count

Period: 5 minutes

InboundMessage

The total number of AS2 messages successfully received from a trading partner.

Units: Count

Period: 5 minutes

InboundFailedMessage

The total number of AS2 messages that were unsuccessfully received from a trading partner. That is, a trading partner sent a message, but the Transfer Family server was not able to successfully process it.

Units: Count

Period: 5 minutes

OnPartialUploadExecutionsStarted

The total number of on-partial-upload workflow executions started on the server.

Units: Count

Period: 1 minute

OnPartialUploadExecutionsSuccess

The total number of successful, on-partial-upload workflow executions on the server.

Units: Count

Period: 1 minute

OnPartialUploadExecutionsFailed

The total number of unsuccessful, on-partial-upload workflow executions on the server.

Units: Count

Period: 1 minute

OnUploadExecutionsStarted

The total number of workflow executions started on the server.

Units: Count

Period: 1 minute

OnUploadExecutionsSuccess

The total number of successful workflow executions on the server.

Units: Count

Period: 1 minute

OnUploadExecutionsFailed

The total number of unsuccessful workflow executions on the server.

Units: Count

Period: 1 minute

Transfer Family dimensions

A dimension is a name/value pair that is part of the identity of a metric. For more information about dimensions, see Dimensions in the Amazon CloudWatch User Guide.

The following table describes the CloudWatch dimension for Transfer Family.

Dimension Description

ServerId

The unique ID of the server.

Using AWS User Notifications with AWS Transfer Family

To get notified about AWS Transfer Family events, you can use AWS User Notifications to set up various delivery channels. When an event matches a rule that you specify, you receive a notification.

You can receive notifications for events through multiple channels, including email, AWS Chatbot chat notifications, or AWS Console Mobile Application push notifications. You can also see notifications in the Console Notifications Center. User Notifications supports aggregation, which can reduce the number of notifications that you receive during specific events.

For more information, see What is AWS User Notifications? in the AWS User Notifications User Guide.