Amazon Snowball in AWS

Share via:

Post Views: 0

Dear Readers,

In this article,we will see Brief introduction on Snow Ball,Snow Ball Edge and Snow Mobile.

Snowball

The AWS Snowball service uses physical storage devices to transfer large amounts of data between Amazon Simple Storage Service (Amazon S3) and your onsite data storage location at faster-than-internet speeds. By working with AWS Snowball, you can save time and money. Snowball provides powerful interfaces that you can use to create jobs, track data, and track the status of your jobs through to completion.

Snowball devices are physically rugged devices that are protected by the AWS Key Management Service (AWS KMS). They secure and protect your data in transit. Regional shipping carriers transport Snowballs between Amazon S3 and your onsite data storage location. For a list of AWS Regions where the Snowball device is available.

Note

Snowball is intended for transferring large amounts of data. If you want to transfer less than 10 TB of data between your on-premises data centres and Amazon S3, Snowball might not be your most economical choice.

Working of Snow Ball

AWS Snowball uses secure devices and the Snowball Client to accelerate petabyte-scale data transfers into and out of AWS services.

You start by using the AWS Management Console to create one or more jobs to request one or multiple Snowball devices (depending on how much data you need to transfer), and download and install the Snowball Client.

Once the device arrives, connect it to your local network, set the IP address either manually or with DHCP, and use the Client to identify the directories you want to copy.

The Client will automatically encrypt and copy the data to the device and notify you when the transfer job is complete.

When the transfer is complete and the device is ready to be returned, the E Ink shipping label will automatically update to indicate the correct AWS partner location to ship to, and you can track the job status by using Amazon Simple Notification Service (SNS), text messages, or directly in the console.

Snowball Client

The Snowball Client is software that you install on a local host computer and use to efficiently identify, compress, encrypt, and transfer data from the directories you specify to a Snowball.

Key Points

You can use the Snowball Client to estimate the time it takes to transfer your data (refer to the user guide for more details).

Data transfer speed is affected by a number of factors including local network speed, file size, and the speed at which data can be read from your local servers.

The Snowball Client will copy data to the Snowball as fast as conditions allow, (as little as a day to copy 48TB of data, depending on your local environment).

End-to-end time to transfer the data into AWS services is approximately a week, including the usual shipping and handling time in AWS service data centres.

You can copy twice that much data in the same amount of time by using two Snowballs in parallel, or you could copy 70TB of data in two and a half days on a single 80TB Snowball, which would increase your end-to-end time to about a week and a half.

Snowball Features

AWS Snowball with the Snowball device has the following features:

80 TB and 50 TB models are available in US Regions; 50 TB model available in all other AWS Regions.

Enforced encryption protects your data at rest and in physical transit.

There’s no need to buy or maintain your own hardware devices.

You can manage your jobs through the AWS Snowball Management Console or programmatically with the job management API.

You can perform local data transfers between your on-premises data centre and a Snowball. You can do these transfers through the Snowball client, a standalone downloadable client. Or you can transfer programmatically using Amazon S3 REST API calls with the downloadable Amazon S3 Adapter for Snowball. The Snowball is its own shipping container, and its E Ink display changes to show your shipping label when the Snowball is ready to ship.

Note

Snowball doesn’t support international shipping or shipping between AWS Regions outside of the US. For more information on shipping restrictions.

Snowball Edge

The AWS Snowball Edge is a type of Snowball device with on-board storage and compute power for select AWS capabilities. Snowball Edge can undertake local processing and edge-computing workloads in addition to transferring data between your local environment and the AWS Cloud.

Each Snowball Edge device can transport data at speeds faster than the internet. This transport is done by shipping the data in the appliances through a regional carrier. The appliances are rugged shipping containers, complete with E Ink shipping labels. The AWS Snowball Edge device differs from the standard Snowball because it can bring the power of the AWS Cloud to your on-premises location, with local storage and compute functionality.

Snowball Edge devices have three options for device configurations – storage optimized, compute optimized, and with GPU. When this guide refers to Snowball Edge devices, it’s referring to all options of the device. Whenever specific information applies only to one or more optional configurations of devices, like how the Snowball Edge with GPU has an on-board GPU.

AWS Snowball Edge Features

Snowball Edge devices have the following features:

Large amounts of storage capacity or compute functionality for devices, depending on the options you choose when you create your job.

Network adapters with transfer speeds of up to 100 GB/second.

Encryption is enforced, protecting your data at rest and in physical transit.

You can import or export data between your local environments and Amazon S3, physically transporting the data with one or more devices, completely bypassing the internet.

AWS Snowball Edge devices are their own rugged shipping containers, and the built-in E Ink display changes to show your shipping label when the device is ready to ship.

Snowball Edge devices come with an on-board LCD display that can be used to manage network connections and get service status information.

You can cluster Snowball Edge devices for local storage and compute jobs to achieve 99.999 percent data durability across 5–10 devices, and to locally grow and shrink storage on demand.

You can use the file interface to read and write data to an AWS Snowball Edge device through a file share or Network File System (NFS) mount point.

You can write Python-language Lambda functions and associate them with Amazon S3 buckets when you create an AWS Snowball Edge device job. Each function triggers whenever there’s a local Amazon S3 PUT object action executed on the associated bucket on the appliance.

Snowball Edge devices have Amazon S3 and Amazon EC2 compatible endpoints available, enabling programmatic use cases.

Snowball Edge devices support the new sbe1, sbe-c, and sbe-g instance types, which you can use to run compute instances on the device using Amazon Machine Images (AMIs).

Snowball Edge Device Options

Snowball Edge devices have three options for device configurations:

Snowball Edge Storage Optimized– This option of Snowball Edge device has the most storage capacity at up to 80 TB of useable storage space, 24 vCPUs, and 32 GiB of memory for compute functionality.

Snowball Edge Compute Optimized– This option of Snowball Edge device has the most compute functionality with 52 vCPUs, 208 GiB of memory, and 7.68 TB of dedicated NVMe SSD storage for instances. This option also comes with 42 TB of additional storage space.

Snowball Edge Compute Optimized with GPU– This option of Snowball Edge devices is identical to the compute optimized option, save for an installed GPU, equivalent to the one available in the P3 Amazon EC2 instance type.

Regional Limitations for AWS Snowball

The AWS Snowball service has two device types, the standard Snowball and the Snowball Edge. The following table highlights which of these devices are available in which regions.

AWS Snowball Use Case Differences

Following is a table that shows the different use cases for the different AWS Snowball devices:

AWS Snowball Hardware Differences

Each device has different storage capacities, as follows:

Snowball

50 TB (42 TB usable) only available in US regions

80 TB (72 TB usable)

Snowball Edge Storage Optimized

100 TB (80 TB usable)

Snowball Edge Compute Optimized

42 TB (39.5 usable) plus 7.68 TB of dedicated NVMe SSD for compute instances

Snowball Edge Compute Optimized with GPU

42 TB (39.5 usable) plus 7.68 TB of dedicated NVMe SSD for compute instances

Each device has the following physical interfaces for management purposes:

Differences Between Items Provided for the Snowball and Snowball Edge

The following outlines the differences between the network adapters, cables used, and cables provided for the Snowball and Snowball Edge.

Snow Mobile

AWS Snowmobile is an Exabyte-scale data transfer service used to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. Snowmobile makes it easy to move massive volumes of data to the cloud, including video libraries, image repositories, or even a complete data center migration. Transferring data with Snowmobile is more secure, fast and cost effective.

After an initial assessment, a Snowmobile will be transported to your data center and AWS personnel will configure it for you so it can be accessed as a network storage target. When your Snowmobile is on site, AWS personnel will work with your team to connect a removable, high-speed network switch from Snowmobile to your local network and you can begin your high-speed data transfer from any number of sources within your data center to the Snowmobile. After your data is loaded, Snowmobile is driven back to AWS where your data is imported into Amazon S3.

Snowmobile uses multiple layers of security to help protect your data including dedicated security personnel, GPS tracking, alarm monitoring, 24/7 video surveillance, and an optional escort security vehicle while in transit. All data is encrypted with 256-bit encryption keys you manage through the AWS Key Management Service(KMS) and designed for security and full chain-of-custody of your data.

Features

Fast transfer even at massive scale

Even with high-speed internet connections, it can take decades to transfer extremely large amounts of data. With Snowmobile, you can move 100 petabytes of data in as little as a few weeks, plus transport time. That same transfer could take more than 20 years to accomplish over a direct connect line with a 1Gbps connection.

Strong encryption

Your data is encrypted with keys that you provide before it is written to Snowmobile. All data is encrypted with 256-bit encryption and you can manage your encryption keys with AWS Key Management Service (KMS). Encryption keys used by encrypting servers on the Snowmobile are never written to disk. Should power be removed for any reason, the keys are securely erased.

Rugged, durable, & more secure

Snowmobile is tamper-resistant, water-resistant and temperature controlled. The data container is only operated by AWS personnel and physical access is limited via security access hardware controls. Snowmobile is protected by 24/7 video surveillance and alarm monitoring, GPS tracking and may optionally be escorted by a security vehicle while in transit.

Customized for your needs

As physical sites can have different migration requirements, AWS will work with you to help ensure all your needs are factored in before Snowmobile arrives on site.

Massively scalable

One Snowmobile can transport up to one hundred petabytes of data in a single trip, the equivalent of using about 1,250 AWS Snowball devices.

Easy data retrieval

Many organizations are concerned that once they have moved all their data to the cloud that it will be both expensive and time-consuming to retrieve the data if needed. Snowball and Snowmobile offer all customers a fast and inexpensive way to ensure data can be quickly transferred both into and out of AWS.