Tutorial: Create a REST API as an Amazon S3 proxy in API Gateway
As an example to showcase using a REST API in API Gateway to proxy Amazon S3, this section describes how to create and configure a REST API to expose the following Amazon S3 operations:
-
Expose GET on the API's root resource to list all of the Amazon S3 buckets of a caller.
-
Expose GET on a Folder resource to view a list of all of the objects in an Amazon S3 bucket.
-
Expose PUT on a Folder resource to add a bucket to Amazon S3.
-
Expose DELETE on a Folder resource to remove a bucket from Amazon S3.
-
Expose GET on a Folder/Item resource to view or download an object from an Amazon S3 bucket.
-
Expose PUT on a Folder/Item resource to upload an object to an Amazon S3 bucket.
-
Expose HEAD on a Folder/Item resource to get object metadata in an Amazon S3 bucket.
-
Expose DELETE on a Folder/Item resource to remove an object from an Amazon S3 bucket.
Note
To integrate your API Gateway API with Amazon S3, you must choose a region where both the API Gateway and Amazon S3 services are available. For region availability, see Amazon API Gateway Endpoints and Quotas.
You may want to import the sample API as an Amazon S3 proxy, as shown in OpenAPI definitions of the sample API as an Amazon S3 proxy. For instructions on how to import an API using the OpenAPI definition, see Configuring a REST API using OpenAPI.
To use the API Gateway console to create the API, you must first sign up for an AWS account. Complete the steps in Prerequisites for getting started with API Gateway.
Topics
- Set up IAM permissions for the API to invoke Amazon S3 actions
- Create API resources to represent Amazon S3 resources
- Expose an API method to list the caller's Amazon S3 buckets
- Expose API methods to access an Amazon S3 bucket
- Expose API methods to access an Amazon S3 object in a bucket
- Call the API using a REST API client
- OpenAPI definitions of the sample API as an Amazon S3 proxy
Set up IAM permissions for the API to invoke Amazon S3 actions
To allow the API to invoke required Amazon S3 actions, you must have appropriate IAM policies attached to an IAM role. The next section describes how to verify and to create, if necessary, the required IAM role and policies.
For your API to view or list Amazon S3 buckets and objects, you can use the IAM-provided AmazonS3ReadOnlyAccess
policy in the IAM role. The ARN of this policy is arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess
,
which is as shown as follows:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:Get*", "s3:List*" ], "Resource": "*" } ] }
This policy document states that any of the Amazon S3 Get*
and List*
actions can be
invoked on any of the Amazon S3 resources.
For your API to update Amazon S3 buckets and objects , you can use a custom policy for any of the Amazon S3
Put*
actions as shown as follows:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:Put*", "Resource": "*" } ] }
For your API to work with Amazon S3 Get*
, List*
and Put*
actions, you can
add the above read-only and put-only policies to the IAM role.
For your API to invoke the Amazon S3 Post*
actions, you must use an Allow policy for the
s3:Post*
actions in the IAM role. For a complete list of Amazon S3 actions, see Specifying Amazon S3 Permissions in a Policy.
For your API to create, view, update, and delete buckets and objects in Amazon S3, you can use the IAM -provided
AmazonS3FullAccess policy in the IAM role. The ARN is arn:aws:iam::aws:policy/AmazonS3FullAccess
.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": "*" } ] }
Having chosen the desired IAM policies to use, create an IAM role and attach to it the policies. The resulting IAM role must contain the following trust policy for API Gateway to assume this role at runtime.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "apigateway.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
When using the IAM console to create the role, choose the Amazon API Gateway role type to ensure that this trust policy is automatically included.
Create API resources to represent Amazon S3 resources
We will use the API's root (/
) resource as the container of an authenticated caller's Amazon S3
buckets. We will also create a Folder
and Item
resources to represent a particular Amazon S3
bucket and a particular Amazon S3 object, respectively. The folder name and object key will be specified, in the form
of path parameters as part of a request URL, by the caller.
Note
When accessing objects whose object key includes /
or any other special character, the
character needs to be URL encoded. For example, test/test.txt
should be encoded to
test%2Ftest.txt
.
To create an API resource that exposes the Amazon S3 service features
-
In the API Gateway console, create an API named MyS3. This API's root resource (/) represents the Amazon S3 service.
-
Under the API's root resource, create a child resource named Folder and set the required Resource Path as /{folder}.
-
For the API's Folder resource, create an Item child resource. Set the required Resource Path as /{item}.
Expose an API method to list the caller's Amazon S3 buckets
Getting the list of Amazon S3 buckets of the caller involves invoking the GET Service action on Amazon S3. On the API's root resource, (/), create the GET method. Configure the GET method to integrate with the Amazon S3, as follows.
To create and initialize the API's GET /
method
-
Choose Create method on the root node (
/
) from the Actions drop-down menu at the top-right corner of the Resources panel. -
Choose the
GET
from the drop-down list of HTTP verbs, and choose the check-mark icon to start creating the method. -
In the / - GET - Setup pane, choose AWS Service for Integration type.
-
From the list, choose a region (e.g.,
us-west-2
) for AWS Region. -
From AWS Service, choose S3.
-
For AWS Subdomain, leave it blank.
-
From HTTP method, choose GET.
-
For Action Type, choose Use path override. With path override, API Gateway forwards the client request to Amazon S3 as the corresponding Amazon S3 REST API path-style request, in which a Amazon S3 resource is expressed by the resource path of the
s3-host-name/bucket/key
pattern. API Gateway sets thes3-host-name
and passes the client specifiedbucket
andkey
from the client to Amazon S3. -
(Optional) In Path override type /.
-
Copy the previously created IAM role's ARN (from the IAM console) and paste it into Execution role.
-
Leave any other settings as default.
-
Choose Save to finish setting up this method.
This setup integrates the frontend GET
https://
request with the
backend your-api-host
/stage
/GET https://
.your-s3-host
/
Note
After the initial setup, you can modify these settings in the Integration Request page of the method.
To control who can call this method of our API, we turn on the method authorization flag and set it to AWS_IAM
.
To enable IAM to control access to the GET / method
-
From the Method Execution, choose Method Request.
-
Choose the pencil icon next to Authorization
-
Choose
AWS_IAM
from the drop-down list. -
Choose the check-mark icon to save the setting.
For our API to return successful responses and exceptions properly to the caller, let us declare the 200, 400 and 500 responses in Method Response. We use the default mapping for 200 responses so that backend responses of the status code not declared here will be returned to the caller as 200 ones.
To declare response types for the GET / method
-
From the Method Execution pane, choose the Method Response box. The API Gateway declares the 200 response by default.
-
Choose Add response, enter
400
in the input text box, and choose the check-mark to finish the declaration. -
Repeat the above step to declare the 500 response type. The final setting is shown as follows:
Because the successful integration response from Amazon S3 returns the bucket list as an XML payload and the
default method response from API Gateway returns a JSON payload, we must map the backend Content-Type header parameter
value to the frontend counterpart. Otherwise, the client will receive application/json
for the content type when the response body is actually an XML string. The following
procedure shows how to set this up. In addition, we also want to display to the client other header parameters,
such as Date and Content-Length.
To set up response header mappings for the GET / method
-
In the API Gateway console, choose Method Response. Add the Content-Type header for the 200 response type.
-
In Integration Response, for Content-Type, type
integration.response.header.Content-Type
for the method response.With the above header mappings, API Gateway will translate the
Date
header from the backend to theTimestamp
header for the client. -
Still in Integration Response, choose Add integration response, type an appropriate regular expression in the HTTP status regex text box for a remaining method response status. Repeat until all the method response status are covered.
As a good practice, let us test our API we have configured so far.
Test the GET method on the API root resource
-
Go back to Method Execution, choose Test from the Client box.
-
Choose Test in the GET / - Method Test pane. An example result is shown as follows.
Note
To use the API Gateway console to test the API as an Amazon S3 proxy, make sure that the targeted S3 bucket is from a different region from the API's region. Otherwise, you may get a 500 Internal Server Error response. This limitation does not apply to any deployed API.
Expose API methods to access an Amazon S3 bucket
To work with an Amazon S3 bucket, we expose the GET, PUT, and DELETE methods on the /{folder} resource to list objects in a bucket, create a new bucket, and delete an existing bucket. The instructions are similar to those described in Expose an API method to list the caller's Amazon S3 buckets. In the following discussions, we outline the general tasks and highlight relevant differences.
To expose GET, PUT and DELETE methods on a folder resource
-
On the /{folder} node from the Resources tree, create the DELETE, GET and PUT methods, one at a time.
-
Set up the initial integration of each created method with its corresponding Amazon S3 endpoints. The following screen shot illustrates this setting for the
PUT /{folder}
method. For theDELETE /{folder}
andGET /{folder}
method, replace thePUT
value of HTTP method byDELETE
andGET
, respectively.Notice that we used the
{bucket}
path parameter in the Amazon S3 endpoint URLs to specify the bucket. We will need to map the{folder}
path parameter of the method requests to the{bucket}
path parameter of the integration requests. -
To map
{folder}
to{bucket}
:-
Choose Method Execution and then Integration Request.
-
Expand URL Path Parameters and choose Add path
-
Type
bucket
in the Name column andmethod.request.path.folder
in the Mapped from column. Choose the check-mark icon to save the mapping.
-
-
In Method Request, add the
Content-Type
to the HTTP Request Headers section.This is mostly needed for testing, when using the API Gateway console, when you must specify
application/xml
for an XML payload. -
In Integration Request, set up a mapping for the
Content-Type
header tomethod.request.header.Content-Type
, following the instructions in Expose an API method to list the caller's Amazon S3 buckets. -
To test the
PUT
method, choose Test in the Client box from Method Execution, and enter the following as input to the testing:-
In folder, type a bucket name,
-
For the Content-Type header, type
application/xml
. -
In Request Body, provide the bucket region as the location constraint, declared in an XML fragment as the request payload. For example,
<CreateBucketConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <LocationConstraint>
{region}
</LocationConstraint> </CreateBucketConfiguration>
-
-
Repeat the preceding steps to create and configure the GET and DELETE method on the API's /{folder} resource.
The above examples illustrate how to create a new bucket in the specified region, to view the list of objects in the bucket, and to delete the bucket. Other Amazon S3 bucket operations allow you work with the metadata or properties of the bucket. For example, you can set up your API to call the Amazon S3's PUT /?notification action to set up notifications on the bucket, to call PUT /?acl to set an access control list on the bucket, etc. The API set up is similar, except for that you must append appropriate query parameters to the Amazon S3 endpoint URLs. At run time, you must provide the appropriate XML payload to the method request. The same can be said about supporting the other GET and DELETE operations on a Amazon S3 bucket. For more information on possible Amazon S3 actions on a bucket, see Amazon S3 Operations on Buckets.
Expose API methods to access an Amazon S3 object in a bucket
Amazon S3 supports GET, DELETE, HEAD, OPTIONS, POST and PUT actions to access and manage objects in a given bucket. For the complete list of supported actions, see Amazon S3 Operations on Objects.
In this tutorial, we expose the PUT Object operation,
the
GET Objectoperation, HEAD Object operation, and the DELETE Object operation through the API methods of PUT /{folder}/{item}
, GET /{folder}/{item}
, HEAD /{folder}/{item}
and DELETE /{folder}/{item}
,
respectively.
The API setups for the PUT, GET and DELETE methods on /{folder}/{item}
are the
similar to those on /{folder}
, as prescribed in Expose API methods to access an Amazon S3 bucket. One major difference is that the object-related request path has an additional path parameter of {item}
and this path parameter must be mapped to the integration request path
parameter of {object}
.

The same is true for the GET and DELETE methods.
As an illustration, the following screen shot shows the output when testing the GET method on a {folder}/{item}
resource using the API Gateway console. The request correctly returns the
plain text of ("Welcome to README.txt") as the content of the specified file (README.txt) in the given Amazon S3 bucket
(apig-demo).

To download or upload binary files, which in API Gateway is considered any thing other than utf-8 encoded JSON content, additional API settings are necessary. This is outlined as follows:
To download or upload binary files from S3
-
Register the media types of the affected file to the API's binaryMediaTypes. You can do this in the console:
-
Choose Settings for the API.
-
Under Binary Media Types, choose Add Binary Media Type.
-
Enter the required media type, for example,
image/png
. -
Choose Save Changes to save the setting.
-
-
Add the
Content-Type
(for upload) and/orAccept
(for download) header to the method request to require the client to specify the required binary media type and map them to the integration request. -
Set Content Handling to
Passthrough
in the integration request (for upload) and in a integration response (for download). Make sure that no mapping template is defined for the affected content type. For more information, see Integration Passthrough Behaviors and Select VTL Mapping Templates.
The payload size limit is 10 MB. See API Gateway quotas for configuring and running a REST API.
Make sure that files on Amazon S3 have the correct content types added as the files' metadata. For streamable media
content, Content-Disposition:inline
may also need to be added to the metadata.
For more information about the binary support in API Gateway, see Content type conversions in API Gateway.
Call the API using a REST API client
To provide an end-to-end tutorial, we now show how to call the API using Postman
To call our Amazon S3 proxy API using Postman
-
Deploy or redeploy the API. Make a note of the base URL of the API that is displayed next to Invoke URL at the top of the Stage Editor.
-
Launch Postman.
-
Choose Authorization and then choose
AWS Signature
. Type your IAM user's Access Key ID and Secret Access Key into the AccessKey and SecretKeyinput fields, respectively. Type the AWS region to which your API is deployed in the AWS Region text box. Typeexecute-api
in the Service Name input field.You can create a pair of the keys from the Security Credentials tab from your IAM user account in the IAM Management Console.
-
To add a bucket named
apig-demo-5
to your Amazon S3 account in the
region:{region}
Note
Be sure that the bucket name must be globally unique.
-
Choose PUT from the drop-down method list and type the method URL (
https://
api-id
.execute-api.aws-region
.amazonaws.com/stage
/folder-name
-
Set the
Content-Type
header value asapplication/xml
. You may need to delete any existing headers before setting the content type. -
Choose Body menu item and type the following XML fragment as the request body:
<CreateBucketConfiguration> <LocationConstraint>
{region}
</LocationConstraint> </CreateBucketConfiguration> -
Choose Send to submit the request. If successful, you should receive a
200 OK
response with an empty payload.
-
-
To add a text file to a bucket, follow the instructions above. If you specify a bucket name of
apig-demo-5
for{folder}
and a file name ofReadme.txt
for{item}
in the URL and provide a text string ofHello, World!
as the file contents (thereby making it the request payload), the request becomesPUT /S3/apig-demo-5/Readme.txt HTTP/1.1 Host: 9gn28ca086.execute-api.
{region}
.amazonaws.com Content-Type: application/xml X-Amz-Date: 20161015T062647Z Authorization: AWS4-HMAC-SHA256 Credential=access-key-id
/20161015/{region}
/execute-api/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=ccadb877bdb0d395ca38cc47e18a0d76bb5eaf17007d11e40bf6fb63d28c705b Cache-Control: no-cache Postman-Token: 6135d315-9cc4-8af8-1757-90871d00847e Hello, World!If everything goes well, you should receive a
200 OK
response with an empty payload. -
To get the content of the
Readme.txt
file we just added to theapig-demo-5
bucket, do a GET request like the following one:GET /S3/apig-demo-5/Readme.txt HTTP/1.1 Host: 9gn28ca086.execute-api.
{region}
.amazonaws.com Content-Type: application/xml X-Amz-Date: 20161015T063759Z Authorization: AWS4-HMAC-SHA256 Credential=access-key-id
/20161015/{region}
/execute-api/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=ba09b72b585acf0e578e6ad02555c00e24b420b59025bc7bb8d3f7aed1471339 Cache-Control: no-cache Postman-Token: d60fcb59-d335-52f7-0025-5bd96928098aIf successful, you should receive a
200 OK
response with theHello, World!
text string as the payload. -
To list items in the
apig-demo-5
bucket, submit the following request:GET /S3/apig-demo-5 HTTP/1.1 Host: 9gn28ca086.execute-api.
{region}
.amazonaws.com Content-Type: application/xml X-Amz-Date: 20161015T064324Z Authorization: AWS4-HMAC-SHA256 Credential=access-key-id
/20161015/{region}
/execute-api/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=4ac9bd4574a14e01568134fd16814534d9951649d3a22b3b0db9f1f5cd4dd0ac Cache-Control: no-cache Postman-Token: 9c43020a-966f-61e1-81af-4c49ad8d1392If successful, you should receive a
200 OK
response with an XML payload showing a single item in the specified bucket, unless you added more files to the bucket before submitting this request.<?xml version="1.0" encoding="UTF-8"?> <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"> <Name>apig-demo-5</Name> <Prefix></Prefix> <Marker></Marker> <MaxKeys>1000</MaxKeys> <IsTruncated>false</IsTruncated> <Contents> <Key>Readme.txt</Key> <LastModified>2016-10-15T06:26:48.000Z</LastModified> <ETag>"65a8e27d8879283831b664bd8b7f0ad4"</ETag> <Size>13</Size> <Owner> <ID>06e4b09e9d...603addd12ee</ID> <DisplayName>
user-name
</DisplayName> </Owner> <StorageClass>STANDARD</StorageClass> </Contents> </ListBucketResult>
Note
To upload or download an image, you need to set content handling to CONVERT_TO_BINARY.