Using the AWS CLI Using the Java SDK Bucket ARN format Bucket location format Creating an S3 bucket on a Snowball Edge device Creating and managing an object lifecycle configuration using the AWS CLI Determining whether you can access an Amazon S3 compatible storage on Snow Family devices bucket Retrieving a list of buckets or regional buckets Getting a bucket Deleting a bucket

Working with S3 buckets with Amazon S3 compatible storage on Snow Family devices

With Amazon S3 compatible storage on Snow Family devices, you can create Amazon S3 buckets on your Snowball Edge devices to store and retrieve objects on premises for applications that require local data access, local data processing, and data residency. Amazon S3 compatible storage on Snow Family devices provides a new storage class, SNOW, which uses the Amazon S3 APIs, and is designed to store data durably and redundantly across multiple Snowball Edge devices. You can use the same APIs and features on Snowball Edge buckets that you do on Amazon S3, including bucket lifecycle policies, encryption, and tagging.

You can use Amazon S3 compatible storage on Snow Family devices using the AWS Command Line Interface (AWS CLI) or programatically through the AWS Java SDK. With the AWS CLI, you can set up an s3api or s3control endpoint and interact with it through commands. We recommend using the s3api endpoint because the same endpoint can be used for bucket and object operations.

Note

The s3api endpoint is available for version 8004 and newer of the Snowball Edge software. To find the version of the Snowball Edge software installed on a device, use the snowballEdge check-for-updates command. To update a Snowball Edge device, see Updating software on Snowball Edge devices.

Using the AWS CLI

Follow these instructions to work with Amazon S3 buckets on your device using the AWS CLI.

To set up the AWS CLI

Create a profile for object endpoints in ~/.aws/config.


[profile your-profile]
aws_access_key_id = your-access-id
aws_secret_access_key = your-access-key
region = snow
ca_bundle = dev/apps/ca-certs/your-ca_bundle

Obtain a certificate from your device. For information, see the Snowball Edge Developer Guide.
If you installed the SDK in a virtual environment, activate it using the following command:
```
source your-virtual-environment-name/bin/activate
```

After you set up your operations, you can use the s3api SDK or s3control SDK to access S3 buckets on Snow Family devices with the AWS CLI.

Example of accessing S3 bucket using the s3api SDK



aws s3api --profile your-profile list-buckets --endpoint-url https://s3api-endpoint-ip

Example of accecssing S3 buckets using the s3control SDK



aws s3control --profile your-profile list-regional-buckets --account-id bucket-owner --endpoint-url https://s3ctrlapi-endpoint-ip

Example of accessing S3 objects using the s3api SDK



aws s3api --profile your-profile list-objects-v2 --endpoint-url https://s3api-endpoint-ip

Using the Java SDK

Use the following example to work with Amazon S3 buckets and objects using the Java SDK.



import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.http.SdkHttpClient;bg
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.regions.Region;

import java.net.URI;

AwsBasicCredentials creds = AwsBasicCredentials.create(accessKey, secretKey); // set creds by getting Access Key and Secret Key from snowball edge
SdkHttpClient httpClient = ApacheHttpClient.builder().tlsTrustManagersProvider(trustManagersProvider).build(); // set trust managers provider with client certificate from snowball edge
String s3SnowEndpoint = "10.0.0.0"; // set s3-snow object api endpoint from describe service

S3Client s3Client = S3Client.builder().httpClient(httpClient).region(Region.of("snow")).endpointOverride(new URI(s3SnowEndpoint)).credentialsProvider(StaticCredentialsProvider.create(creds)).build();

Bucket ARN format

You can use the Amazon Resource Name (ARN) format listed here to identify an Amazon S3 bucket on a Snowball Edge device:


arn:partition:s3:snow:account-id:device/device-id/bucket/bucket-name

Where partition is the partition of the Region where you ordered your Snowball Edge device. device-id is the job_id if the device is a standalone Snowball Edge device, or the cluster_id if you have a Snowball Edge cluster.

Bucket location format

The bucket location format specifies the Snowball Edge device where the bucket will be createed. The bucket location has the following format:


/device-id/bucket/bucket-name

For more information, see create-bucket in the AWS CLI Command Reference.

Creating an S3 bucket on a Snowball Edge device

You can create Amazon S3 buckets on your Snowball Edge device to store and retrieve objects at the edge for applications that require local data access, local data processing, and data residency. Amazon S3 compatible storage on Snow Family devices provides a new storage class, SNOW, which uses Amazon S3 and is designed to store data durably and redundantly across multiple devices . You can use the same APIs and features as you do on Amazon S3 buckets, including bucket lifecycle policies, encryption, and tagging.

The following example creates an Amazon S3 bucket for a Snowball Edge device using the AWS CLI. To run this command, replace the user input placeholders with your own information.

Example of creating an S3 bucket

Creating and managing an object lifecycle configuration using the AWS CLI

You can use Amazon S3 Lifecycle to optimize storage capacity for Amazon S3 compatible storage on Snow Family devices. You can create lifecycle rules to expire objects as they age or are replaced by newer versions. You can create, enable, disable, or delete a lifecycle rule. For more information about Amazon S3 Lifecycle, see Managing your storage lifecycle.

Note

The AWS account that creates the bucket owns it and is the only one that can create, enable, disable, or delete a lifecycle rule.

To create and manage a lifecycle configuration for an Amazon S3 compatible storage on Snow Family devices bucket using the AWS Command Line Interface (AWS CLI), see the following examples.

PUT a lifecycle configuration on a Snowball Edge bucket

The following AWS CLI example puts a lifecycle configuration policy on a Snowball Edge bucket. This policy specifies that all objects that have the flagged prefix (myprefix) and tags expire after 10 days. To use this example, replace each user input placeholder with your own information.

First, save the lifecycle configuration policy to a JSON file. For this example, the file is named lifecycle-example.json.


{
    "Rules": [{
        "ID": "id-1",
        "Filter": {
            "And": {
                "Prefix": "myprefix",
                "Tags": [{
                        "Value": "mytagvalue1",
                        "Key": "mytagkey1"
                    },
                    {
                        "Value": "mytagvalue2",
                        "Key": "mytagkey2"
                    }
                ]
            }
        },
        "Status": "Enabled",
        "Expiration": {
            "Days": 10
        }
    }]
}

After you save the file, submit the JSON file as part of the put-bucket-lifecycle-configuration command. To use this command, replace each user input placeholder with your own information.

Example of `put-bucket-lifecycle` command

Determining whether you can access an Amazon S3 compatible storage on Snow Family devices bucket

The following example uses the head-bucket command to determine if an Amazon S3 bucket exists and you have permissions to access it using the AWS CLI. To use this command, replace each user input placeholder with your own information.


aws s3api head-bucket --bucket sample-bucket --profile your-profile --endpoint-url https://s3api-endpoint-ip

Retrieving a list of buckets or regional buckets

Use the list-regional-buckets or list-buckets to list Amazon S3 compatible storage on Snow Family devices buckets using the AWS CLI.

Example of retrieving a list of buckets or regional buckets with AWS CLI

The following SDK for Java example gets a list of buckets on Snowball Edge devices. For more information, see ListBuckets in the Amazon Simple Storage Service API Reference.



  import com.amazonaws.services.s3.model.*;
  public void listBuckets() {
    ListBucketsRequest reqListBuckets = new ListBucketsRequest()
    .withAccountId(AccountId)
    ListBucketsResult respListBuckets = s3APIClient.RegionalBuckets(reqListBuckets);
    System.out.printf("ListBuckets Response: %s%n", respListBuckets.toString());
  }

The following PowerShell example gets a list of buckets on Snowball Edge devices.



   Get-S3CRegionalBucketList -AccountId 012345678910 -Endpoint "https://snowball_ip" -Region snow

The following .NET example gets a list of buckets on Snowball Edge devices.



using Amazon.S3Control;
using Amazon.S3Control.Model;

namespace SnowTest;

internal class Program
{
    static async Task Main(string[] args)
    {
        var config = new AmazonS3ControlConfig
        {
            ServiceURL = "https://snowball_ip",
            AuthenticationRegion = "snow" // Note that this is not RegionEndpoint
        };        

        var client = new AmazonS3ControlClient(config);

        var response = await client.ListRegionalBucketsAsync(new ListRegionalBucketsRequest() 
        { 
            AccountId = "012345678910" 
        });
    }
}

Getting a bucket

The following example gets an Amazon S3 compatible storage on Snow Family devices bucket using the AWS CLI. To use this command, replace each user input placeholder with your own information.


aws s3control get-bucket --account-id 123456789012 --bucket amzn-s3-demo-bucket --profile your-profile --endpoint-url https://s3ctrlapi-endpoint-ip

For more information about this command, see get-bucket in the AWS CLI Command Reference.

The following Amazon S3 compatible storage on Snow Family devices example gets a bucket using the SDK for Java. For more information, see GetBucket in the Amazon Simple Storage Service API Reference.



import com.amazonaws.services.s3control.model.*;

public void getBucket(String bucketName) {

    GetBucketRequest reqGetBucket = new GetBucketRequest()
            .withBucket(bucketName)
            .withAccountId(AccountId);

    GetBucketResult respGetBucket = s3ControlClient.getBucket(reqGetBucket);
    System.out.printf("GetBucket Response: %s%n", respGetBucket.toString());
}

Deleting a bucket

You can use the s3api SDK or s3control SDK to delete a bucket in Amazon S3 compatible storage on Snow Family devices.

Important

The AWS account that creates the bucket owns it and is the only one that can delete it.
Snow Family devices buckets must be empty before they can be deleted.
You cannot recover a bucket after it has been deleted.

The following examples delete an Amazon S3 compatible storage on Snow Family devices bucket using the AWS CLI. To use this command, replace each user input placeholder with your own information.

Example of deleting a bucket

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Setting up and starting Amazon S3 compatible storage on Snow Family devices

Working with S3 objects on a Snowball Edge device

Working with S3 buckets with Amazon S3 compatible storage on Snow Family devices

Note

Using the AWS CLI

To set up the AWS CLI

Example of accessing S3 bucket using the s3api SDK

Example of accecssing S3 buckets using the s3control SDK

Example of accessing S3 objects using the s3api SDK

Using the Java SDK

Bucket ARN format

Bucket location format

Creating an S3 bucket on a Snowball Edge device

Example of creating an S3 bucket

Creating and managing an object lifecycle configuration using the AWS CLI

Note

PUT a lifecycle configuration on a Snowball Edge bucket

Example of put-bucket-lifecycle command

Determining whether you can access an Amazon S3 compatible storage on Snow Family devices bucket

Retrieving a list of buckets or regional buckets

Example of retrieving a list of buckets or regional buckets with AWS CLI

Getting a bucket

Deleting a bucket

Important

Example of deleting a bucket

Example of `put-bucket-lifecycle` command