Menu
Amazon Kinesis Firehose API Reference
API Reference (API Version 2015-08-04)

CreateDeliveryStream

Creates a delivery stream.

By default, you can create up to 20 delivery streams per region.

This is an asynchronous operation that immediately returns. The initial status of the delivery stream is CREATING. After the delivery stream is created, its status is ACTIVE and it now accepts data. Attempts to send data to a delivery stream that is not in the ACTIVE state cause an exception. To check the state of a delivery stream, use DescribeDeliveryStream.

A Kinesis Firehose delivery stream can be configured to receive records directly from providers using PutRecord or PutRecordBatch, or it can be configured to use an existing Kinesis stream as its source. To specify a Kinesis stream as input, set the DeliveryStreamType parameter to KinesisStreamAsSource, and provide the Kinesis stream ARN and role ARN in the KinesisStreamSourceConfiguration parameter.

A delivery stream is configured with a single destination: Amazon S3, Amazon ES, or Amazon Redshift. You must specify only one of the following destination configuration parameters: ExtendedS3DestinationConfiguration, S3DestinationConfiguration, ElasticsearchDestinationConfiguration, or RedshiftDestinationConfiguration.

When you specify S3DestinationConfiguration, you can also provide the following optional values: BufferingHints, EncryptionConfiguration, and CompressionFormat. By default, if no BufferingHints value is provided, Kinesis Firehose buffers data up to 5 MB or for 5 minutes, whichever condition is satisfied first. Note that BufferingHints is a hint, so there are some cases where the service cannot adhere to these conditions strictly; for example, record boundaries are such that the size is a little over or under the configured buffering size. By default, no encryption is performed. We strongly recommend that you enable encryption to ensure secure data storage in Amazon S3.

A few notes about Amazon Redshift as a destination:

  • An Amazon Redshift destination requires an S3 bucket as intermediate location, as Kinesis Firehose first delivers data to S3 and then uses COPY syntax to load data into an Amazon Redshift table. This is specified in the RedshiftDestinationConfiguration.S3Configuration parameter.

  • The compression formats SNAPPY or ZIP cannot be specified in RedshiftDestinationConfiguration.S3Configuration because the Amazon Redshift COPY operation that reads from the S3 bucket doesn't support these compression formats.

  • We strongly recommend that you use the user name and password you provide exclusively with Kinesis Firehose, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.

Kinesis Firehose assumes the IAM role that is configured as part of the destination. The role should allow the Kinesis Firehose principal to assume the role, and the role should have permissions that allow the service to deliver the data. For more information, see Amazon S3 Bucket Access in the Amazon Kinesis Firehose Developer Guide.

Request Syntax

Copy
{ "DeliveryStreamName": "string", "DeliveryStreamType": "string", "ElasticsearchDestinationConfiguration": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "DomainARN": "string", "IndexName": "string", "IndexRotationPeriod": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3BackupMode": "string", "S3Configuration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "Prefix": "string", "RoleARN": "string" }, "TypeName": "string" }, "ExtendedS3DestinationConfiguration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "Prefix": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RoleARN": "string", "S3BackupConfiguration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "Prefix": "string", "RoleARN": "string" }, "S3BackupMode": "string" }, "KinesisStreamSourceConfiguration": { "KinesisStreamARN": "string", "RoleARN": "string" }, "RedshiftDestinationConfiguration": { "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "ClusterJDBCURL": "string", "CopyCommand": { "CopyOptions": "string", "DataTableColumns": "string", "DataTableName": "string" }, "Password": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3BackupConfiguration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "Prefix": "string", "RoleARN": "string" }, "S3BackupMode": "string", "S3Configuration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "Prefix": "string", "RoleARN": "string" }, "Username": "string" }, "S3DestinationConfiguration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "Prefix": "string", "RoleARN": "string" } }

Request Parameters

The request accepts the following data in JSON format.

DeliveryStreamName

The name of the delivery stream. This name must be unique per AWS account in the same region. If the delivery streams are in different accounts or different regions, you can have multiple delivery streams with the same name.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [a-zA-Z0-9_.-]+

Required: Yes

DeliveryStreamType

The delivery stream type. This parameter can be one of the following values:

  • DirectPut: Provider applications access the delivery stream directly.

  • KinesisStreamAsSource: The delivery stream uses a Kinesis stream as a source.

Type: String

Valid Values: DirectPut | KinesisStreamAsSource

Required: No

ElasticsearchDestinationConfiguration

The destination in Amazon ES. You can specify only one destination.

Type: ElasticsearchDestinationConfiguration object

Required: No

ExtendedS3DestinationConfiguration

The destination in Amazon S3. You can specify only one destination.

Type: ExtendedS3DestinationConfiguration object

Required: No

KinesisStreamSourceConfiguration

When a Kinesis stream is used as the source for the delivery stream, a KinesisStreamSourceConfiguration containing the Kinesis stream ARN and the role ARN for the source stream.

Type: KinesisStreamSourceConfiguration object

Required: No

RedshiftDestinationConfiguration

The destination in Amazon Redshift. You can specify only one destination.

Type: RedshiftDestinationConfiguration object

Required: No

S3DestinationConfiguration

[Deprecated] The destination in Amazon S3. You can specify only one destination.

Type: S3DestinationConfiguration object

Required: No

Response Syntax

Copy
{ "DeliveryStreamARN": "string" }

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

DeliveryStreamARN

The ARN of the delivery stream.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 512.

Pattern: arn:.*

Errors

For information about the errors that are common to all actions, see Common Errors.

InvalidArgumentException

The specified input parameter has a value that is not valid.

HTTP Status Code: 400

LimitExceededException

You have already reached the limit for a requested resource.

HTTP Status Code: 400

ResourceInUseException

The resource is already in use and not available for this operation.

HTTP Status Code: 400

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following: