AWS services or capabilities described in AWS Documentation may vary by region/location. Click Getting Started with Amazon AWS to see specific differences applicable to the China (Beijing) Region.
New-KINFDeliveryStream-HttpEndpointDestinationConfiguration_EndpointConfiguration_AccessKey <String>-AmazonopensearchserviceDestinationConfiguration_ClusterEndpoint <String>-ElasticsearchDestinationConfiguration_ClusterEndpoint <String>-HttpEndpointDestinationConfiguration_RequestConfiguration_CommonAttributes <HttpEndpointCommonAttribute[]>-HttpEndpointDestinationConfiguration_RequestConfiguration_ContentEncoding <ContentEncoding>-DeliveryStreamName <String>-DeliveryStreamType <DeliveryStreamType>-AmazonopensearchserviceDestinationConfiguration_DomainARN <String>-ElasticsearchDestinationConfiguration_DomainARN <String>-AmazonopensearchserviceDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>-RetryOptions_DurationInSecond <Int32>-HttpEndpointDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>-AmazonopensearchserviceDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>-CloudWatchLoggingOptions_Enabled <Boolean>-ProcessingConfiguration_Enabled <Boolean>-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>-HttpEndpointDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>-ExtendedS3DestinationConfiguration <ExtendedS3DestinationConfiguration>-AmazonopensearchserviceDestinationConfiguration_IndexName <String>-ElasticsearchDestinationConfiguration_IndexName <String>-AmazonopensearchserviceDestinationConfiguration_IndexRotationPeriod <AmazonopensearchserviceIndexRotationPeriod>-ElasticsearchDestinationConfiguration_IndexRotationPeriod <ElasticsearchIndexRotationPeriod>-AmazonopensearchserviceDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>-BufferingHints_IntervalInSecond <Int32>-HttpEndpointDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>-DeliveryStreamEncryptionConfigurationInput_KeyARN <String>-DeliveryStreamEncryptionConfigurationInput_KeyType <KeyType>-KinesisStreamSourceConfiguration_KinesisStreamARN <String>-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>-CloudWatchLoggingOptions_LogGroupName <String>-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>-CloudWatchLoggingOptions_LogStreamName <String>-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>-EndpointConfiguration_Name <String>-AmazonopensearchserviceDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>-ProcessingConfiguration_Processor <Processor[]>-HttpEndpointDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>-RedshiftDestinationConfiguration <RedshiftDestinationConfiguration>-AmazonopensearchserviceDestinationConfiguration_RoleARN <String>-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_RoleARN <String>-ElasticsearchDestinationConfiguration_RoleARN <String>-VpcConfiguration_RoleARN <String>-HttpEndpointDestinationConfiguration_RoleARN <String>-KinesisStreamSourceConfiguration_RoleARN <String>-AmazonopensearchserviceDestinationConfiguration_S3BackupMode <AmazonopensearchserviceS3BackupMode>-ElasticsearchDestinationConfiguration_S3BackupMode <ElasticsearchS3BackupMode>-HttpEndpointDestinationConfiguration_S3BackupMode <HttpEndpointS3BackupMode>-AmazonopensearchserviceDestinationConfiguration_S3Configuration <S3DestinationConfiguration>-ElasticsearchDestinationConfiguration_S3Configuration <S3DestinationConfiguration>-HttpEndpointDestinationConfiguration_S3Configuration <S3DestinationConfiguration>-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_SecurityGroupIds <String[]>-VpcConfiguration_SecurityGroupId <String[]>-AmazonopensearchserviceDestinationConfiguration_BufferingHints_SizeInMBs <Int32>-BufferingHints_SizeInMBs <Int32>-HttpEndpointDestinationConfiguration_BufferingHints_SizeInMBs <Int32>-SplunkDestinationConfiguration <SplunkDestinationConfiguration>-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_SubnetIds <String[]>-VpcConfiguration_SubnetId <String[]>-Tag <Tag[]>-AmazonopensearchserviceDestinationConfiguration_TypeName <String>-ElasticsearchDestinationConfiguration_TypeName <String>-HttpEndpointDestinationConfiguration_EndpointConfiguration_Url <String>-S3DestinationConfiguration <S3DestinationConfiguration>-Select <String>-Force <SwitchParameter>
CREATING
. After the delivery stream is created, its status is ACTIVE
and it now accepts data. If the delivery stream creation fails, the status transitions to CREATING_FAILED
. Attempts to send data to a delivery stream that is not in the ACTIVE
state cause an exception. To check the state of a delivery stream, use DescribeDeliveryStream.
If the status of a delivery stream is CREATING_FAILED
, this status doesn't change, and you can't invoke CreateDeliveryStream
again on it. However, you can invoke the DeleteDeliveryStream operation to delete it.
A Kinesis Data Firehose delivery stream can be configured to receive records directly from providers using PutRecord or PutRecordBatch, or it can be configured to use an existing Kinesis stream as its source. To specify a Kinesis data stream as input, set the DeliveryStreamType
parameter to KinesisStreamAsSource
, and provide the Kinesis stream Amazon Resource Name (ARN) and role ARN in the KinesisStreamSourceConfiguration
parameter.
To create a delivery stream with server-side encryption (SSE) enabled, include DeliveryStreamEncryptionConfigurationInput in your request. This is optional. You can also invoke StartDeliveryStreamEncryption to turn on SSE for an existing delivery stream that doesn't have SSE enabled.
A delivery stream is configured with a single destination: Amazon S3, Amazon ES, Amazon Redshift, or Splunk. You must specify only one of the following destination configuration parameters: ExtendedS3DestinationConfiguration
, S3DestinationConfiguration
, ElasticsearchDestinationConfiguration
, RedshiftDestinationConfiguration
, or SplunkDestinationConfiguration
.
When you specify S3DestinationConfiguration
, you can also provide the following optional values: BufferingHints, EncryptionConfiguration
, and CompressionFormat
. By default, if no BufferingHints
value is provided, Kinesis Data Firehose buffers data up to 5 MB or for 5 minutes, whichever condition is satisfied first. BufferingHints
is a hint, so there are some cases where the service cannot adhere to these conditions strictly. For example, record boundaries might be such that the size is a little over or under the configured buffering size. By default, no encryption is performed. We strongly recommend that you enable encryption to ensure secure data storage in Amazon S3.
A few notes about Amazon Redshift as a destination: COPY
syntax to load data into an Amazon Redshift table. This is specified in the RedshiftDestinationConfiguration.S3Configuration
parameter. SNAPPY
or ZIP
cannot be specified in RedshiftDestinationConfiguration.S3Configuration
because the Amazon Redshift COPY
operation that reads from the S3 bucket doesn't support these compression formats. INSERT
permissions. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
ec2:DescribeVpcs
ec2:DescribeVpcAttribute
ec2:DescribeSubnets
ec2:DescribeSecurityGroups
ec2:DescribeNetworkInterfaces
ec2:CreateNetworkInterface
ec2:CreateNetworkInterfacePermission
ec2:DeleteNetworkInterface
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_BufferingHints_IntervalInSeconds |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_BufferingHints_SizeInMBs |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_CloudWatchLoggingOptions_Enabled |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName |
KeyType
to CUSTOMER_MANAGED_CMK
, you must specify the Amazon Resource Name (ARN) of the CMK. If you set KeyType
to AWS_OWNED_CMK
, Kinesis Data Firehose uses a service-account CMK. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
AWS_OWNED_CMK
. For more information about CMKs, see Customer Master Keys (CMKs). When you invoke CreateDeliveryStream or StartDeliveryStreamEncryption with KeyType
set to CUSTOMER_MANAGED_CMK, Kinesis Data Firehose invokes the Amazon KMS operation CreateGrant to create a grant that allows the Kinesis Data Firehose service to use the customer managed CMK to perform encryption and decryption. Kinesis Data Firehose manages that grant. When you invoke StartDeliveryStreamEncryption to change the CMK for a delivery stream that is encrypted with a customer managed CMK, Kinesis Data Firehose schedules the grant it had on the old CMK for retirement.You can use a CMK of type CUSTOMER_MANAGED_CMK to encrypt up to 500 delivery streams. If a CreateDeliveryStream or StartDeliveryStreamEncryption operation exceeds this limit, Kinesis Data Firehose throws a LimitExceededException
.Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | True |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
DirectPut
: Provider applications access the delivery stream directly.KinesisStreamAsSource
: The delivery stream uses a Kinesis data stream as a source.Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
ClusterEndpoint
or the DomainARN
field. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
DescribeElasticsearchDomain
, DescribeElasticsearchDomains
, and DescribeElasticsearchDomainConfig
after assuming the role specified in RoleARN. For more information, see Amazon Resource Names (ARNs) and AWS Service Namespaces.Specify either ClusterEndpoint
or DomainARN
. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
IndexName
to facilitate the expiration of old data. For more information, see Index Rotation for the Amazon ES Destination. The default value is OneDay
. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
FailedDocumentsOnly
, Kinesis Data Firehose writes any documents that could not be indexed to the configured Amazon S3 destination, with elasticsearch-failed/
appended to the key prefix. When set to AllDocuments
, Kinesis Data Firehose delivers all incoming records to Amazon S3, and also writes failed documents with elasticsearch-failed/
appended to the prefix. For more information, see Amazon S3 Backup for the Amazon ES Destination. Default value is FailedDocumentsOnly
.You can't change this backup mode after you create the delivery stream. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
TypeName
. Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | HttpEndpointDestinationConfiguration_EndpointConfiguration_Name |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
AllData
) or only the documents that Kinesis Data Firehose could not deliver to the specified HTTP endpoint destination (FailedDataOnly
). Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_ProcessingConfiguration_Enabled |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_ProcessingConfiguration_Processors |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_RetryOptions_DurationInSeconds |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | Tags |
ec2:DescribeVpcs
ec2:DescribeVpcAttribute
ec2:DescribeSubnets
ec2:DescribeSecurityGroups
ec2:DescribeNetworkInterfaces
ec2:CreateNetworkInterface
ec2:CreateNetworkInterfacePermission
ec2:DeleteNetworkInterface
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_VpcConfiguration_RoleARN |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_VpcConfiguration_SecurityGroupIds |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ElasticsearchDestinationConfiguration_VpcConfiguration_SubnetIds |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | AK |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByValue, ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByValue, ByPropertyName) |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | AWSProfilesLocation, ProfilesLocation |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | StoredCredentials, AWSProfileName |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | RegionToCall |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | SK, SecretAccessKey |
Required? | False |
Position? | Named |
Accept pipeline input? | True (ByPropertyName) |
Aliases | ST |
AWS Tools for PowerShell: 2.x.y.z