| « PreviousNext » | |
   | Did this page help you? Yes | No | Tell us about it... |
Introduction to Amazon S3
Topics
This introduction to Amazon S3 is intended to give you a detailed summary of this web service. After reading this section, you should have a good idea of what it offers and how you can use Amazon S3 for your business.
Overview of Amazon S3
Amazon S3 is storage for the Internet. It is designed to make web-scale computing easier for developers.
The AWS Management Console makes it easy to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any user access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites.
Advantages to Amazon S3
Amazon S3 is intentionally built with a minimal feature set that focuses on simplicity and robustness. Following are some of advantages of the Amazon S3 service:
Unlimited storage—There is no limit to the amount of data you can store on Amazon S3
Standard interface—Amazon S3 uses standards based REST and SOAP interfaces designed to work with any Internet-development toolkit
Scalable—Amazon S3 can scale in terms of storage, request rate, and users to support an unlimited number of web-scale applications
Reliability—Store data with up to 99.999999999% durability, with 99.99% availability
Inexpensive—Amazon S3 is built from inexpensive commodity hardware components
Amazon S3 Concepts
This section describes key concepts and terminology you need to understand to use Amazon S3 effectively. They are presented in the order you will most like encounter them.
Buckets
A bucket is a container for objects stored in Amazon S3. Every object is contained in
a bucket. For example, if the object named photos/puppy.jpg is stored
in the johnsmith bucket, then it is addressable using the URL
http://johnsmith.s3.amazonaws.com/photos/puppy.jpg
Buckets serve several purposes: they organize the Amazon S3 namespace at the highest level, they identify the account responsible for storage and data transfer charges, they play a role in access control, and they serve as the unit of aggregation for usage reporting.
You can configure buckets so that they are created in a specific Region. For more information, see Regions.
Objects
Objects are the fundamental entities stored in Amazon S3. When using the console, you can think of them as being files. Objects consist of data and metadata. The data portion is opaque to Amazon S3. The metadata is a set of name-value pairs that describe the object. These include some default metadata such as the date last modified, and standard HTTP metadata such as Content-Type. The developer can also specify custom metadata at the time the object is stored.
An object is uniquely identified within a bucket by a key (name).
Folders
Folders are available in the AWS Management Console, but are not part of the core Amazon S3 API. You use folders to group objects in a bucket.
When you create a folder in the AWS Management Console, Amazon S3 creates a
zero-byte object with a forward slash (/) at the end of the object name in your
bucket. Amazon S3 interprets the forward slash as a delimiter when performing list
operations. For example, if you create a new folder in the AWS Management Console
called logs, Amazon S3 creates an object called logs/. If you upload an object called history.txt to the logs folder using the AWS
Management Console, the full key name for this object is logs/history.txt.
For more information about how Amazon S3 treats keys, go to Amazon S3 Developer Guide.
Keys
A key is like a file name; it is the unique identifier for an object within a bucket. Every object in a bucket has exactly one key. Because the combination of a bucket, key, and version ID uniquely identify each object, Amazon S3 can be thought of as a basic data map between "bucket + key + version" and the object itself. Every object in Amazon S3 can be uniquely addressed through the combination of the web service endpoint, bucket name, key, and optionally, a version. For example, in the URL http://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl, "doc" is the name of the bucket and "2006-03-01/AmazonS3.wsdl" is the key.
Regions
You can choose the geographical Region where Amazon S3 will store the buckets you create. You might choose a Region to optimize latency, minimize costs, or address regulatory requirements. Amazon S3 currently supports the following Regions:
US Standard—Uses Amazon S3 servers in the United States
This is the default Region. The US Standard Region automatically routes requests to facilities in Northern Virginia or the Pacific Northwest using network maps. To use this region, select US - Standard as the region when creating a bucket in the console. The US Standard Region provides eventual consistency for all requests.
US West (Oregon) Region—Uses Amazon S3 servers in Oregon
To use this Region, choose Oregon as the Region when creating the bucket in the AWS Management Console. In Amazon S3, the US West (Oregon) Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
US West (Northern California) Region—Uses Amazon S3 servers in Northern California
To use this Region, choose US - N. California as the Region when creating the bucket in the AWS Management Console. In Amazon S3, the US Northern California Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
EU (Ireland) Region—Uses Amazon S3 servers in Ireland
To use this Region, choose EU - Ireland as the Region when creating the bucket in the AWS Management Console.. In Amazon S3, the EU (Ireland) Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
Asia Pacific (Singapore) Region—Uses Amazon S3 servers in Singapore
To use this Region, choose Singapore as the Region when creating the bucket in the AWS Management Console. In Amazon S3, the Asia Pacific (Singapore) Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
Asia Pacific (Sydney) Region—Uses Amazon S3 servers in Sydney
To use this Region, choose Sydney as the Region when creating the bucket in the AWS Management Console. In Amazon S3, the Asia Pacific (Sydney) Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
Asia Pacific (Tokyo) Region—Uses Amazon S3 servers in Tokyo
To use this Region, choose Tokyo as the Region when creating the bucket in the AWS Management Console. In Amazon S3, the Asia Pacific (Tokyo) Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
South America (Sao Paulo) Region—Uses Amazon S3 servers in Sao Paulo
To use this Region, choose Sao Paulo as the Region when creating the bucket in the AWS Management Console. In Amazon S3, the South America (Sao Paulo) Region provides read-after-write consistency for PUTS of new objects in your Amazon S3 bucket and eventual consistency for overwrite PUTS and DELETES.
Objects stored in a Region never leave the Region unless you explicitly transfer them to another Region. For example, objects stored in the EU (Ireland) Region never leave it. The objects stored in an S3 region physically remain in that region. Amazon S3 does not keep copies or move it to any other region. However, you can access the objects from anywhere, as long as you have necessary permissions.
Access Control
Amazon S3 has two ways of controlling access to buckets and objects: access control lists (ACLs) and bucket policies. Access Control Lists (ACLs), you can define the permissions associated with each individual Amazon S3 bucket or object resource. Policies are a collection of statements that define a user’s permissions to access Amazon S3 resources. Policies can be attached to users, groups, or Amazon S3 buckets, enabling centralized management of permissions.
Amazon S3 Data Consistency Model
Updates to a single key are atomic. For example, if you PUT
to an existing key, a subsequent read might return the old data or the updated data,
but it will never write corrupted or partial data.
Amazon S3 achieves high availability by replicating data across multiple servers within Amazon's data centers. After a "success" is returned, your data is safely stored. However, information about the changes might not immediately replicate across Amazon S3 and you might observe the following behaviors:
A process writes a new object to Amazon S3 and immediately attempts to read it. Until the change is fully propagated, Amazon S3 might report "key does not exist."
A process writes a new object to Amazon S3 and immediately lists keys within its bucket. Until the change is fully propagated, the object might not appear in the list.
A process replaces an existing object and immediately attempts to read it. Until the change is fully propagated, Amazon S3 might return the prior data.
A process deletes an existing object and immediately attempts to read it. Until the deletion is fully propagated, Amazon S3 might return the deleted data.
A process deletes an existing object and immediately lists keys within its bucket. Until the deletion is fully propagated, Amazon S3 might list the deleted object.
The US Standard Region provides eventual consistency for all requests. All other regions provide read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES.
Note
Amazon S3 does not currently support object locking. If two puts are simultaneously made to the same key, the put with the latest time stamp wins. If this is an issue, you will need to build an object-locking mechanism into your application.
Updates are key-based; there is no way to make atomic updates across keys. For example, you cannot make the update of one key dependent on the update of another key unless you design this functionality into your application.
The following table describes the characteristics of eventually consistent read and consistent read.
| Eventually Consistent Read | Consistent Read |
|---|---|
| Stale reads possible | No stale reads |
| Lowest read latency | Potential higher read latency |
| Highest read throughput | Potential lower read throughput |
For more information about the Amazon S3 Data Consistency Model see the Amazon S3 Developer Guide.
Limitations of the AWS Management Console
The AWS Management Console is a powerful tool that makes using Amazon S3 easy. The following features are currently unavailable in the AWS Management Console:
Requester pays
BitTorrent
Versioning
The AWS Management Console will be updated to support all these Amazon S3 features.
Paying for Amazon S3
Pricing for Amazon S3 is designed so that you don't have to plan for the storage requirements of your application. Most storage providers force you to purchase a predetermined amount of storage and network transfer capacity: If you exceed that capacity, your service is shut off or you are charged high overage fees. If you do not exceed that capacity, you pay as though you used it all.
Amazon S3 charges you only for what you actually use, with no hidden fees and no overage charges. This gives developers a variable-cost service that can grow with their business while enjoying the cost advantages of Amazon's infrastructure.
Before storing anything in Amazon S3, you need to register with the service and provide a payment instrument that will be charged at the end of each month. There are no set-up fees to begin using the service. At the end of the month, your payment instrument is automatically charged for that month's usage.
For information about paying for Amazon S3 storage, go to the AWS Resource Center.
Related Amazon Web Service Products
Once we load your data into Amazon S3 you can use it with all AWS products. The following products are the ones you might use most frequently:
Amazon CloudFront—This web service provides an easy way to distribute content to end users with low latency, high data transfer speeds, and no commitments.
For more information, go to Amazon CloudFront.
Amazon ElasticCompute Cloud—This web service provides virtual compute resources in the cloud.
For more information, go to Amazon Elastic Compute Cloud.
Amazon Elastic MapReduce—This web service enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data.
It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3). For more information, go to Amazon Elastic MapReduce.
Amazon Import/Export—This service enables you to mail a storage device, such as a RAID drive, to Amazon so that we can upload your (terabytes) of data onto Amazon S3. For more information, go to AWS Import/Export Developer Guide.
