How AWS Snowcone Works - AWS Snowcone User Guide

How AWS Snowcone Works

AWS Snowcone is a portable device used for edge computing and data transfer. To get started, you request one or more Snowcone devices in the AWS Management Console based on how much data you need to transfer and the compute performance required. The Amazon Simple Storage Service (Amazon S3) buckets, data, and Amazon Elastic Compute Cloud (Amazon EC2) Amazon Machine Images (AMIs) that you choose are automatically configured, encrypted, and pre-installed on your devices. The AWS DataSync agent is also pre-installed before your devices are shipped to you.

You typically receive your Snowcone devices within approximately 4–6 days. To receive multiple AWS Snowcone devices, you must set up a job order on the console for each Snowcone device.

When your device arrives, you connect it to your on-premises network and set the IP address either manually or automatically with Dynamic Host Configuration Protocol (DHCP). You must download and install AWS OpsHub for Snow Family, a graphical user interface (GUI) application for managing your Snowcone device. You can install it on any Windows or macOS client machine, such as a laptop.

When you open AWS OpsHub and unlock the device, you see a dashboard showing your device and its system metrics. You can then launch instances to deploy your edge applications or migrate your data to the device with just a few clicks in AWS OpsHub.

When your compute or data transfer job is completed and the device is ready to be returned, the E Ink shipping label automatically updates the return address, ensuring that the Snowcone device is delivered to the correct AWS facility. When the device ships, you can receive tracking status through messages sent by Amazon Simple Notification Service (Amazon SNS), generated texts and emails, or directly from the console.

AWS Snowcone Workflow

You can create three different job types. Although the job types differ in their use cases, they all have the following workflow for ordering, receiving, and returning the device.

The workflow

  1. Create the job – You create each job on the AWS Snow Family Management Console or programmatically through the job management API and choose a device type—Snowcone or Snowcone SSD—depending on your use case. You can track the status of the job on the AWS Management Console or through the Snowcone API.

  2. A device is prepared for your job – AWS prepares an AWS Snowcone device for your job, and the status of your job changes to Preparing Snowcone.

  3. A device is shipped to you by your Region's carrier – The shipping carrier takes over from here, and the status of your job now changes to In transit to you. You can find your tracking number and a link to the tracking website on the AWS Snow Family Management Console console or with the job management API. For information about who your Region's carrier is, see Shipping Considerations for AWS Snowcone.

  4. Get a Snowcone power supply – To maintain the smallest footprint, Snowcone devices do not ship with a power supply. Snowcone uses a 45 watt USB-C connected power supply. It can also be powered by a portable battery. For more information, see AWS Snowcone Power Supply and Accessories.

  5. Receive the device – A few days later, your Region's shipping carrier delivers the AWS Snowcone device to the address that you provided when you created the job. The status of your job changes to Delivered to you. The device does not arrive in a box because the device is its own shipping container.

  6. Get your credentials and download the AWS OpsHub or Snowball Edge client for Snow Family application – Get ready to start transferring data by getting your credentials, your job manifest, and the manifest's unlock code, and then downloading the Snowball Edge client.

    • Get the manifest for your device from the console or with the job management API when the device is on-premises at your location. The manifest is used to authenticate your access to the device. The manifest is encrypted, so only the unlock code can decrypt it.

    • The unlock code is a 29-character code used to decrypt the manifest. You can get the unlock code from the console or with the job management API. To prevent unauthorized access to the device while it's at your facility, we recommend that you keep the unlock code in a safe location that is different from the location of the manifest.

    • AWS OpsHub for Snow Family is an application for managing Snow Family Devices, including Snowcone. The AWS OpsHub for Snow Family GUI helps you set up and manage Snowcone devices so that you can quickly run your edge compute workloads and migrate data to the AWS Cloud. With just a few clicks, you can use AWS OpsHub to unlock and configure your Snowcone device, drag and drop data, launch applications, or monitor device metrics. You can download and install it on Windows or macOS client machines, such as a laptop. There is no cost to use AWS OpsHub.

      Download AWS OpsHub from AWS Snowball resources. For more information about AWS OpsHub, see Using AWS OpsHub for Snow Family to Manage Devices.

    • The Snowball Edge client is the tool that you use to manage the flow of data from the device to your on-premises data destination.

  7. Position the hardware – Move the device into your data center and open it following the instructions on the case. Connect the device to a power supply and your local network.

  8. Power on the device – Power on the device by pressing the power button above the LCD display. Wait a few minutes, and the Ready screen appears.

  9. Get the IP address for the device – The LCD display has a CONNECTION tab on it. Tap this tab and get the IP address for the AWS Snowcone device.

  10. Use AWS OpsHub to unlock the device – To unlock the AWS Snowcone device, you enter the IP address of the device, upload your manifest, and the unlock code. AWS OpsHub decrypts the manifest and uses it to authenticate your access to the device. For more information about AWS OpsHub, see Using AWS OpsHub for Snow Family to Manage Devices.

  11. Use the device – Use AWS OpsHub to set up and manage AWS Snowcone devices so that you can quickly run your edge compute workloads and transfer data to the AWS Cloud. With just a few clicks, you can use AWS OpsHub to unlock and configure your Snowcone device, drag and drop data, launch applications, or monitor device metrics. For details, see Using AWS OpsHub for Snow Family to Manage Devices.

  12. Don't unplug the Ethernet or power supply cables – Don't unplug the Ethernet cable or the USB-C power supply or battery during data transfer or computing operations. To turn off the Snowcone device after your data transfer or compute job is complete, press the power button.

  13. Prepare the device for its return trip – After you're done with the device in your on-premises location, and the file interface status changes to Complete, press the power button above the LCD display to power off the device. Unplug the device and store its power cables in the cable nook on top of the device, and shut all three of the device's doors. The device is now ready to be returned.

  14. Your Region's carrier returns the device to AWS – When the carrier has the AWS Snowcone device, the status for the job changes to In transit to AWS.

How Import and Export Jobs Work

You can use Snowcone to transfer data online between your device and AWS storage services by using AWS DataSync. You can also transfer data offline from your on-premises storage devices to your Snowcone device.

Online Data Transfer Between Snowcone and AWS Storage Services with DataSync

AWS DataSync is an online data transfer service that simplifies, automates, and accelerates copying large amounts of data to and from AWS storage services over the internet or AWS Direct Connect. An AWS DataSync agent is pre-installed on your Snowcone device and is used to transfer data between the device and Amazon S3 buckets, Amazon Elastic File System (Amazon EFS) file systems, and Amazon FSx for Windows File Server. DataSync automatically handles moving files and objects, scheduling data transfers, monitoring the progress of transfers, encrypting data, verifying data transfers, and notifying customers of any issues.

The DataSync agent is preinstalled on your Snowcone device as an Amazon Machine Image (AMI) during the Snowcone job preparation. To transfer data online to AWS, connect the Snowcone device to the external network and use AWS OpsHub for Snow Family or the AWS Command Line Interface (AWS CLI) to launch the DataSync agent AMI. Activate the DataSync agent using the AWS Management Console or the AWS CLI. Then set up your online data transfer task between the AWS Snowcone Network File System (NFS) store and Amazon S3, Amazon EFS, or Amazon FSx.

You can use DataSync running on Snowcone for the following:

  • For edge computing applications, to collect data, process the data to gain immediate insight, and then transfer the data online to AWS.

  • To transfer data that is continuously generated by sensors or machines online to AWS in a factory or at other edge locations.

  • To distribute media, scientific, or other content online from AWS storage services to your partners and customers.

  • To aggregate content by transferring media, scientific, or other content online from your edge locations to AWS.

For one-time edge compute or data transfer workflows or for Snowcone workflows in edge locations without a wide area network (WAN) link or inadequate WAN bandwidth, we recommend shipping the Snowcone device back to AWS to complete the data transfer.

Offline Data Transfer Between Snowcone and Amazon S3

For offline data import jobs, you connect the Snowcone device to your on-premises network and then use AWS OpsHub to unlock the device. Download AWS OpsHub from the AWS Snowball resources page. You can copy data from on-premises storage devices to your Snowcone device through the NFS file interface. After you copy the data to your Snowcone device, the E Ink shipping label on the device helps ensure that the device is automatically sent to the correct AWS facility. You can track the Snowcone device by using Amazon SNS–generated text messages or emails and the console.

Exporting Jobs from Amazon S3

Your data source for an export job is one or more Amazon S3 buckets. After the data for a job part is moved from Amazon S3 to a Snowcone device, you can download a job report. This report alerts you to any objects that failed the transfer to the device. You can find more information in your job's success and failure logs.

You can export up to 50 million objects for each export job, using as many devices as it takes to complete the transfer. If you need to transfer more than 50 million objects, you must create multiple export jobs. Each Snowcone device for an export job's job parts is delivered one after another, with subsequent devices shipping to you after the previous job part enters the In transit to AWS status.

When you copy objects into your on-premises data destination from a device, those objects are saved as files. If you copy objects into a location that already holds files, any existing files with the same names are overwritten.

When AWS receives a returned device, we completely erase it, following National Institute of Standards and Technology (NIST) 800-88 standards.

Important

Don't change, update, or delete the exported Amazon S3 objects until you can verify that all of your contents for the entire job have been copied to your on-premises data destination.

When you create an export job, you can export an entire Amazon S3 bucket or a specific range of object keys.

How Export Jobs Work

Each export job can use any number of Snowcone devices. If the listing contains more data than can fit on a single device, multiple devices are provided to you. Each job part has exactly one device associated with it. After your job parts are created, your first job part enters the Preparing Snowcone status.

Note

The listing operation used to split your job into parts is a function of Amazon S3, and you are billed for it the same way as any Amazon S3 operation.

Soon after that, we start exporting your data onto a device. Typically, exporting data takes one business day. However, this process can take longer depending on the amount and type of data. When the export is done, AWS gets the device ready for pickup by your Region's carrier. When it arrives, you connect the Snowcone device to your network and transfer the data that you want to import from Amazon S3 onto the device.

When you're done transferring data, ship the device back to AWS. When we receive the device for your export job part, we erase it completely. This erasure follows NIST 800-88 standards. This step marks the completion of that particular job part.

Important
  • For keylisting

    Before we export the objects in the S3 bucket, we scan the bucket. If the bucket is altered after the scan, the job could encounter delays because we scan for missing or altered objects.

  • For Amazon S3 Glacier

    Snowcone cannot export objects in the S3 Glacier storage class. These objects must be restored to S3 buckets before Snowcone can successfully export the objects in the bucket.

Using Export Ranges

When you create an export job in the AWS Snow Family Management Console or with the job management API, you can export an entire Amazon S3 bucket or a specific range of object keys. Object key names uniquely identify objects in a bucket. If you export a range, you define the length of the range by providing either an inclusive range beginning, an inclusive range ending, or both.

Ranges are UTF-8 binary sorted. UTF-8 binary data is sorted in the following way:

  • The numbers 0–9 come before both uppercase and lowercase English characters.

  • Uppercase English characters come before all lowercase English characters.

  • Lowercase English characters come last when sorted against uppercase English characters and numbers.

  • Special characters are sorted among the other character sets.

For more information about the specifics of UTF-8, see UTF-8 on Wikipedia.

Export Range Examples

Assume that you have a bucket containing the following objects, sorted in UTF-8 binary order:

  • 01

  • Aardvark

  • Aardwolf

  • Aasvogel/apple

  • Aasvogel/banana

  • Aasvogel/cherry

  • Banana

  • Car

Specified range beginning Specified range ending Objects in the range that will be exported
(none) (none) All of the objects in your bucket
(none) Aasvogel

01

Aardvark

Aardwolf

Aasvogel/apple

Aasvogel/banana

Aasvogel/cherry

(none) Aasvogel/banana

01

Aardvark

Aardwolf

Aasvogel/apple

Aasvogel/banana

Aasvogel (none)

Aasvogel/apple

Aasvogel/banana

Aasvogel/cherry

Banana

Car

Aardwolf (none)

Aardwolf

Aasvogel/apple

Aasvogel/banana

Aasvogel/cherry

Banana

Car

Aar (none)

Aardvark

Aardwolf

Aasvogel/apple

Aasvogel/banana

Aasvogel/cherry

Banana

Car

car (none)

No objects are exported, and you get an error message when you try to create the job. Note that car is sorted after Car according to UTF-8 binary values.

Aar Aarrr

Aardvark

Aardwolf

For Import Job Storage

Internally, a Snowcone device contains 8 TB or 14 TB of disk storage that can be used with the internal Network File System (NFS) service or local Amazon EC2 instances through a local Amazon Elastic Block Store (Amazon EBS) volume presentation. You can use 8 TB or 14 TB for the NFS storage volume and 150 GB for the capacity-optimized HDD (sbg1) Amazon EBS storage volume.

For Compute Job Storage

If the job type is local compute, you might create a total of 8 TB or 14 TB of local capacity-optimized HDD (sbg1) Amazon EBS volumes and attach them to Amazon EC2 instances. Using Amazon EBS volumes allows the local Amazon EC2 instances to access more local capacity than the root volume alone. Because this is local storage only, data written to the Amazon EBS volumes is lost when the device is returned to AWS because it can't be imported into Amazon S3.

Note

The NFS server is not available for compute jobs. If you need to import or export data to or from the AWS Cloud, don't choose the local compute job type when you place your order.