Amazon Simple Storage Service
Developer Guide (API Version 2006-03-01)
Did this page help you?  Yes | No |  Tell us about it...
« PreviousNext »
View the PDF for this guide.Go to the AWS Discussion Forum for this product.Go to the Kindle Store to download this guide in Kindle format.

Object Key and Metadata

Each Amazon S3 object has data, a key, and metadata. Object key (or key name) uniquely identifies the object in a bucket. Object metadata is a set of name-value pairs. You can set object metadata at the time you upload it. After you upload the object, you cannot modify object metadata. The only way to modify object metadata is to make a copy of the object and set the metadata.

Object Keys

When you create an object, you specify the key name, which uniquely identifies the object in the bucket. For example, in the Amazon S3 console (see AWS Management Console), when you highlight a bucket, a list of objects in your bucket appears. These names are the object keys. The name for a key is a sequence of Unicode characters whose UTF-8 encoding is at most 1024 bytes long.

Note

If you anticipate that your workload against Amazon S3 will exceed 100 requests per second, follow the Amazon S3 key naming guidelines for best performance. For information, see Request Rate and Performance Considerations.

Object Key Naming Guidelines

Although you can use any UTF-8 characters in an object key name, the following key naming best practices help ensure maximum compatibility with other applications.  Each application may parse special characters differently. The following guidelines help you maximize compliance with DNS, web safe characters, XML parsers, and other APIs.

Safe Characters

The following character sets are generally safe for use in key names:

  • Alphanumeric characters [0-9a-zA-Z]

  • Special characters !, -, _, ., *, ', (, and )

The following are examples of valid object key names:

  • 4my-organization

  • my.great_photos-2014/jan/myvacation.jpg

  • videos/2014/birthday/video1.wmv

Note that the Amazon S3 data model is a flat structure: you create a bucket, and the bucket stores objects. There is no hierarchy of subbuckets or subfolders; however, you can infer logical hierarchy using keyname prefixes and delimiters as the Amazon S3 console does. The Amazon S3 console supports a concept of folders. Suppose your bucket (companybucket) has four objects with the following object keys:

Development/Projects1.xls

Finance/statement1.pdf

Private/taxdocument.pdf

s3-dg.pdf

The console uses the keyname prefixes (Development/, Finance/, and Private/) and delimiter ('/') to present a folder structure as shown:

The s3-dg.pdf key does not have a prefix, so its object appears directly at the root level of the bucket. If you open the Development/ folder, you will see the Project1.xls object in it.

Note

Amazon S3 supports buckets and objects, there is no hierarchy in Amazon S3. However, the prefixes and delimiters in an object key name, enables the Amazon S3 console and the AWS SDKs to infer hierarchy and introduce concept of folders.

Characters That Might Require Special Handling

The following characters in a key name may require additional code handling and will likely need to be URL encoded or referenced as HEX. Some of these are non-printable characters and your browser may not handle them, which will also require special handling:

Ampersand ("&")

Dollar ("$")

ASCII character ranges 00–1F hex (0–31 decimal) and 7F (127 decimal.)

'At' symbol ("@")

Equals ("=")

Semicolon (";")

Colon (":")

Plus ("+")

Space – Significant sequences of spaces may be lost in some uses (especially multiple spaces)

Comma (",")

Question mark ("?")

 

Characters to Avoid

You should avoid the following characters in a key name because of significant special handling for consistency across all applications.

Backslash ("\")

Left curly brace ("{")

Non-printable ASCII characters (128–255 decimal characters)

Caret ("^")

Right curly brace ("}")

Percent character ("%")

Grave accent / back tick ("`")

Right square bracket ("]")

Quotation marks

'Greater Than' symbol (">")

Left square bracket ("[")

Tilde ("~")

'Less Than' symbol ("<")

'Pound' character ("#")

Vertical bar / pipe ("|")

Object Metadata

There are two kinds of metadata: system metadata and user-defined metadata.

System-Defined Metadata

For each object stored in a bucket, Amazon S3 maintains a set of system metadata. Amazon S3 processes this system metadata as needed. For example, Amazon S3 maintains object creation date and size metadata and uses this information as part of object management.

There are two categories of system metadata:

  • Metadata such as object creation date is system controlled where only Amazon S3 can modify the value.

  • Other system metadata such as the storage class configured for the object and whether the object has server-side encryption enabled are examples of system metadata whose values you control. If you have your bucket configured as a website, sometimes you might want to redirect a page request to another page or an external URL. In this case, a web page is an object in your bucket. Amazon S3 stores the page redirect value as system metadata whose value you control.

    When you create objects, you can configure values of these system metadata items or update the values when you need. For more information about storage class and server-side encryption, see Protecting Data Using Encryption.

The following table provides a list of system-defined metadata and whether you can update it.

NameDescriptionCan User Modify the Value?
DateObject creation date.No
Content-LengthObject size in bytes.No
Last-ModifiedDate the object was last modified.No
Content-MD5The base64-encoded 128-bit MD5 digest of the object.No
x-amz-server-side-encryptionIndicates whether server-side encryption is enabled for the object, and whether that encryption is from the AWS Key Management Service (SSE-KMS) or from AWS-Managed Encryption (SSE-S3). For more information, see Protecting Data Using Server-Side Encryption. Yes
x-amz-version-idObject version. When you enable versioning on a bucket, Amazon S3 assigns a version number to objects added to the bucket. For more information, see Using Versioning.No
x-amz-delete-markerIn a bucket that has versioning enabled, this Boolean marker indicates whether the object is a delete marker. No
x-amz-storage-classStorage class used for storing the object. Yes
x-amz-website-redirect-location Redirects requests for the associated object to another object in the same bucket or an external URL. For more information, see Configuring a Web Page Redirect .Yes
x-amz-server-side-encryption-aws-kms-key-idIf the x-amz-server-side-encryption is present and has the value of aws:kms, this indicates the ID of the Key Management Service (KMS) master encryption key that was used for the object.Yes
x-amz-server-side-encryption-customer-algorithmIndicates whether server-side encryption with customer-provided encryption keys (SSE-C) is enabled. For more information, see Protecting Data Using Server-Side Encryption with Customer-Provided Encryption Keys (SSE-C). Yes

Storage Classes

Each Amazon S3 object is associated with a storage class. Amazon S3 supports the following storage classes:

  • Standard — The Standard storage class provides 99.999999999% durability and 99.99% availability of objects over a given year. It is designed to sustain the concurrent loss of data in two facilities.

  • Reduced Redundancy Storage (RRS) — The RRS storage class reduces costs by storing noncritical, reproducible data at lower levels of redundancy than the Standard storage class. It provides 99.99% durability and 99.99% availability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.01% of objects

  • GLACIER — The GLACIER storage class is suitable for archiving data, where data access is infrequent and a retrieval time of several hours is acceptable. The GLACIER storage class uses the very low-cost Amazon Glacier storage service, but you still manage objects in this storage class through Amazon S3.

Note

You cannot associate an object with the GLACIER storage class as you upload it. You transition existing Amazon S3 objects to the GLACIER storage class by using lifecycle management. For more information, see Object Lifecycle Management.

User-Defined Metadata

When uploading an object, you can also assign metadata to the object. You provide this optional information as a name-value pair when you send a PUT or POST request to create the object. When uploading objects using the REST API the optional user-defined metadata names must begin with "x-amz-meta-" to distinguish them from other HTTP headers. When you retrieve the object using the REST API, this prefix is returned. When uploading objects using the SOAP API, the prefix is not required. When you retrieve the object using the SOAP API, the prefix is removed, regardless of which API you used to upload the object.

Note

SOAP support over HTTP is deprecated, but it is still available over HTTPS. New Amazon S3 features will not be supported for SOAP. We recommend that you use either the REST API or the AWS SDKs.

When metadata is retrieved through the REST API, Amazon S3 combines headers that have the same name (ignoring case) into a comma-delimited list. If some metadata contains unprintable characters, it is not returned. Instead, the "x-amz-missing-meta" header is returned with a value of the number of the unprintable metadata entries.

Amazon S3 stores user-defined metadata in lowercase. Each name, value pair must conform to US-ASCII when using REST and UTF-8 when using SOAP or browser-based uploads via POST.

Note

The PUT request header is limited to 8 KB in size. Within the PUT request header, the user-defined metadata is limited to 2 KB in size. User-defined metadata is a set of key-value pairs. The size of user-defined metadata is measured by taking the sum of the number of bytes in the UTF-8 encoding of each key and value.