AWS Tools for Windows PowerShell
Command Reference

AWS services or capabilities described in AWS Documentation may vary by region/location. Click Getting Started with Amazon AWS to see specific differences applicable to the China (Beijing) Region.

Synopsis

Calls the Amazon Kinesis Firehose CreateDeliveryStream API operation.

Syntax

New-KINFDeliveryStream
-HttpEndpointDestinationConfiguration_EndpointConfiguration_AccessKey <String>
-SnowflakeDestinationConfiguration_AccountUrl <String>
-CatalogConfiguration_CatalogARN <String>
-AmazonopensearchserviceDestinationConfiguration_ClusterEndpoint <String>
-ElasticsearchDestinationConfiguration_ClusterEndpoint <String>
-AmazonOpenSearchServerlessDestinationConfiguration_CollectionEndpoint <String>
-HttpEndpointDestinationConfiguration_RequestConfiguration_CommonAttributes <HttpEndpointCommonAttribute[]>
-AuthenticationConfiguration_Connectivity <Connectivity>
-SnowflakeDestinationConfiguration_ContentColumnName <String>
-HttpEndpointDestinationConfiguration_RequestConfiguration_ContentEncoding <ContentEncoding>
-SnowflakeDestinationConfiguration_Database <String>
-SnowflakeDestinationConfiguration_DataLoadingOption <SnowflakeDataLoadingOption>
-AmazonopensearchserviceDestinationConfiguration_DocumentIdOptions_DefaultDocumentIdFormat <DefaultDocumentIdFormat>
-DocumentIdOptions_DefaultDocumentIdFormat <DefaultDocumentIdFormat>
-DeliveryStreamName <String>
-DeliveryStreamType <DeliveryStreamType>
-IcebergDestinationConfiguration_DestinationTableConfigurationList <DestinationTableConfiguration[]>
-AmazonopensearchserviceDestinationConfiguration_DomainARN <String>
-ElasticsearchDestinationConfiguration_DomainARN <String>
-AmazonOpenSearchServerlessDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
-AmazonopensearchserviceDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
-RetryOptions_DurationInSecond <Int32>
-HttpEndpointDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
-IcebergDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
-SnowflakeDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
-AmazonOpenSearchServerlessDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
-AmazonOpenSearchServerlessDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
-AmazonopensearchserviceDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
-CloudWatchLoggingOptions_Enabled <Boolean>
-ProcessingConfiguration_Enabled <Boolean>
-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
-HttpEndpointDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
-HttpEndpointDestinationConfiguration_SecretsManagerConfiguration_Enabled <Boolean>
-IcebergDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
-IcebergDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
-SnowflakeDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
-SnowflakeDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
-SnowflakeDestinationConfiguration_SecretsManagerConfiguration_Enabled <Boolean>
-SnowflakeRoleConfiguration_Enabled <Boolean>
-ExtendedS3DestinationConfiguration <ExtendedS3DestinationConfiguration>
-AmazonOpenSearchServerlessDestinationConfiguration_IndexName <String>
-AmazonopensearchserviceDestinationConfiguration_IndexName <String>
-ElasticsearchDestinationConfiguration_IndexName <String>
-AmazonopensearchserviceDestinationConfiguration_IndexRotationPeriod <AmazonopensearchserviceIndexRotationPeriod>
-ElasticsearchDestinationConfiguration_IndexRotationPeriod <ElasticsearchIndexRotationPeriod>
-AmazonOpenSearchServerlessDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
-AmazonopensearchserviceDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
-BufferingHints_IntervalInSecond <Int32>
-HttpEndpointDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
-IcebergDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
-SnowflakeDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
-DeliveryStreamEncryptionConfigurationInput_KeyARN <String>
-SnowflakeDestinationConfiguration_KeyPassphrase <String>
-DeliveryStreamEncryptionConfigurationInput_KeyType <KeyType>
-KinesisStreamSourceConfiguration_KinesisStreamARN <String>
-AmazonOpenSearchServerlessDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
-CloudWatchLoggingOptions_LogGroupName <String>
-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
-IcebergDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
-SnowflakeDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
-AmazonOpenSearchServerlessDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
-CloudWatchLoggingOptions_LogStreamName <String>
-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
-IcebergDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
-SnowflakeDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
-SnowflakeDestinationConfiguration_MetaDataColumnName <String>
-MSKSourceConfiguration_MSKClusterARN <String>
-EndpointConfiguration_Name <String>
-SnowflakeDestinationConfiguration_PrivateKey <String>
-SnowflakeVpcConfiguration_PrivateLinkVpceId <String>
-AmazonOpenSearchServerlessDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
-AmazonopensearchserviceDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
-ProcessingConfiguration_Processor <Processor[]>
-HttpEndpointDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
-IcebergDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
-SnowflakeDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
-MSKSourceConfiguration_ReadFromTimestamp <DateTime>
-RedshiftDestinationConfiguration <RedshiftDestinationConfiguration>
-AmazonOpenSearchServerlessDestinationConfiguration_RoleARN <String>
-AmazonOpenSearchServerlessDestinationConfiguration_VpcConfiguration_RoleARN <String>
-AmazonopensearchserviceDestinationConfiguration_RoleARN <String>
-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_RoleARN <String>
-ElasticsearchDestinationConfiguration_RoleARN <String>
-VpcConfiguration_RoleARN <String>
-HttpEndpointDestinationConfiguration_RoleARN <String>
-HttpEndpointDestinationConfiguration_SecretsManagerConfiguration_RoleARN <String>
-IcebergDestinationConfiguration_RoleARN <String>
-KinesisStreamSourceConfiguration_RoleARN <String>
-AuthenticationConfiguration_RoleARN <String>
-SnowflakeDestinationConfiguration_RoleARN <String>
-SnowflakeDestinationConfiguration_SecretsManagerConfiguration_RoleARN <String>
-AmazonOpenSearchServerlessDestinationConfiguration_S3BackupMode <AmazonOpenSearchServerlessS3BackupMode>
-AmazonopensearchserviceDestinationConfiguration_S3BackupMode <AmazonopensearchserviceS3BackupMode>
-ElasticsearchDestinationConfiguration_S3BackupMode <ElasticsearchS3BackupMode>
-HttpEndpointDestinationConfiguration_S3BackupMode <HttpEndpointS3BackupMode>
-IcebergDestinationConfiguration_S3BackupMode <IcebergS3BackupMode>
-SnowflakeDestinationConfiguration_S3BackupMode <SnowflakeS3BackupMode>
-AmazonOpenSearchServerlessDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
-AmazonopensearchserviceDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
-ElasticsearchDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
-HttpEndpointDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
-IcebergDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
-SnowflakeDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
-SnowflakeDestinationConfiguration_Schema <String>
-HttpEndpointDestinationConfiguration_SecretsManagerConfiguration_SecretARN <String>
-SnowflakeDestinationConfiguration_SecretsManagerConfiguration_SecretARN <String>
-AmazonOpenSearchServerlessDestinationConfiguration_VpcConfiguration_SecurityGroupIds <String[]>
-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_SecurityGroupIds <String[]>
-VpcConfiguration_SecurityGroupId <String[]>
-AmazonOpenSearchServerlessDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
-AmazonopensearchserviceDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
-BufferingHints_SizeInMBs <Int32>
-HttpEndpointDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
-IcebergDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
-SnowflakeDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
-SnowflakeRoleConfiguration_SnowflakeRole <String>
-SplunkDestinationConfiguration <SplunkDestinationConfiguration>
-AmazonOpenSearchServerlessDestinationConfiguration_VpcConfiguration_SubnetIds <String[]>
-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_SubnetIds <String[]>
-VpcConfiguration_SubnetId <String[]>
-SnowflakeDestinationConfiguration_Table <String>
-Tag <Tag[]>
-MSKSourceConfiguration_TopicName <String>
-AmazonopensearchserviceDestinationConfiguration_TypeName <String>
-ElasticsearchDestinationConfiguration_TypeName <String>
-HttpEndpointDestinationConfiguration_EndpointConfiguration_Url <String>
-SnowflakeDestinationConfiguration_User <String>
-S3DestinationConfiguration <S3DestinationConfiguration>
-Select <String>
-Force <SwitchParameter>
-ClientConfig <AmazonKinesisFirehoseConfig>

Description

Creates a Firehose delivery stream. By default, you can create up to 50 delivery streams per Amazon Web Services Region. This is an asynchronous operation that immediately returns. The initial status of the delivery stream is CREATING. After the delivery stream is created, its status is ACTIVE and it now accepts data. If the delivery stream creation fails, the status transitions to CREATING_FAILED. Attempts to send data to a delivery stream that is not in the ACTIVE state cause an exception. To check the state of a delivery stream, use DescribeDeliveryStream. If the status of a delivery stream is CREATING_FAILED, this status doesn't change, and you can't invoke CreateDeliveryStream again on it. However, you can invoke the DeleteDeliveryStream operation to delete it. A Firehose delivery stream can be configured to receive records directly from providers using PutRecord or PutRecordBatch, or it can be configured to use an existing Kinesis stream as its source. To specify a Kinesis data stream as input, set the DeliveryStreamType parameter to KinesisStreamAsSource, and provide the Kinesis stream Amazon Resource Name (ARN) and role ARN in the KinesisStreamSourceConfiguration parameter. To create a delivery stream with server-side encryption (SSE) enabled, include DeliveryStreamEncryptionConfigurationInput in your request. This is optional. You can also invoke StartDeliveryStreamEncryption to turn on SSE for an existing delivery stream that doesn't have SSE enabled. A delivery stream is configured with a single destination, such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon OpenSearch Service, Amazon OpenSearch Serverless, Splunk, and any custom HTTP endpoint or HTTP endpoints owned by or supported by third-party service providers, including Datadog, Dynatrace, LogicMonitor, MongoDB, New Relic, and Sumo Logic. You must specify only one of the following destination configuration parameters: ExtendedS3DestinationConfiguration, S3DestinationConfiguration, ElasticsearchDestinationConfiguration, RedshiftDestinationConfiguration, or SplunkDestinationConfiguration. When you specify S3DestinationConfiguration, you can also provide the following optional values: BufferingHints, EncryptionConfiguration, and CompressionFormat. By default, if no BufferingHints value is provided, Firehose buffers data up to 5 MB or for 5 minutes, whichever condition is satisfied first. BufferingHints is a hint, so there are some cases where the service cannot adhere to these conditions strictly. For example, record boundaries might be such that the size is a little over or under the configured buffering size. By default, no encryption is performed. We strongly recommend that you enable encryption to ensure secure data storage in Amazon S3. A few notes about Amazon Redshift as a destination:
  • An Amazon Redshift destination requires an S3 bucket as intermediate location. Firehose first delivers data to Amazon S3 and then uses COPY syntax to load data into an Amazon Redshift table. This is specified in the RedshiftDestinationConfiguration.S3Configuration parameter.
  • The compression formats SNAPPY or ZIP cannot be specified in RedshiftDestinationConfiguration.S3Configuration because the Amazon Redshift COPY operation that reads from the S3 bucket doesn't support these compression formats.
  • We strongly recommend that you use the user name and password you provide exclusively with Firehose, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions.
Firehose assumes the IAM role that is configured as part of the destination. The role should allow the Firehose principal to assume the role, and the role should have permissions that allow the service to deliver the data. For more information, see Grant Firehose Access to an Amazon S3 Destination in the Amazon Firehose Developer Guide.

Parameters

-AmazonOpenSearchServerlessDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300 (5 minutes).
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting this parameter to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec, the value should be 10 MB or higher.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
Enables or disables CloudWatch logging.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
The CloudWatch group name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
The CloudWatch log stream name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_CollectionEndpoint <String>
The endpoint to use when communicating with the collection in the Serverless offering for Amazon OpenSearch Service.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_IndexName <String>
The Serverless offering for Amazon OpenSearch Service index name.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
Enables or disables data processing.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
The data processors.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
After an initial failure to deliver to the Serverless offering for Amazon OpenSearch Service, the total amount of time during which Firehose retries delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. Default value is 300 seconds (5 minutes). A value of 0 (zero) results in no retries.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_RoleARN <String>
The Amazon Resource Name (ARN) of the IAM role to be assumed by Firehose for calling the Serverless offering for Amazon OpenSearch Service Configuration API and for indexing documents.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_S3BackupMode <AmazonOpenSearchServerlessS3BackupMode>
Defines how documents should be delivered to Amazon S3. When it is set to FailedDocumentsOnly, Firehose writes any documents that could not be indexed to the configured Amazon S3 destination, with AmazonOpenSearchService-failed/ appended to the key prefix. When set to AllDocuments, Firehose delivers all incoming records to Amazon S3, and also writes failed documents with AmazonOpenSearchService-failed/ appended to the prefix.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
The service has not provided documentation for this parameter; please refer to the service's API reference documentation for the latest available information.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_VpcConfiguration_RoleARN <String>
The ARN of the IAM role that you want the delivery stream to use to create endpoints in the destination VPC. You can use your existing Firehose delivery role or you can specify a new role. In either case, make sure that the role trusts the Firehose service principal and that it grants the following permissions:
  • ec2:DescribeVpcs
  • ec2:DescribeVpcAttribute
  • ec2:DescribeSubnets
  • ec2:DescribeSecurityGroups
  • ec2:DescribeNetworkInterfaces
  • ec2:CreateNetworkInterface
  • ec2:CreateNetworkInterfacePermission
  • ec2:DeleteNetworkInterface
When you specify subnets for delivering data to the destination in a private VPC, make sure you have enough number of free IP addresses in chosen subnets. If there is no available free IP address in a specified subnet, Firehose cannot create or add ENIs for the data delivery in the private VPC, and the delivery will be degraded or fail.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_VpcConfiguration_SecurityGroupIds <String[]>
The IDs of the security groups that you want Firehose to use when it creates ENIs in the VPC of the Amazon ES destination. You can use the same security group that the Amazon ES domain uses or different ones. If you specify different security groups here, ensure that they allow outbound HTTPS traffic to the Amazon ES domain's security group. Also ensure that the Amazon ES domain's security group allows HTTPS traffic from the security groups specified here. If you use the same security group for both your delivery stream and the Amazon ES domain, make sure the security group inbound rule allows HTTPS traffic. For more information about security group rules, see Security group rules in the Amazon VPC documentation.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonOpenSearchServerlessDestinationConfiguration_VpcConfiguration_SubnetIds <String[]>
The IDs of the subnets that you want Firehose to use to create ENIs in the VPC of the Amazon ES destination. Make sure that the routing tables and inbound and outbound rules allow traffic to flow from the subnets whose IDs are specified here to the subnets that have the destination Amazon ES endpoints. Firehose creates at least one ENI in each of the subnets that are specified here. Do not delete or modify these ENIs.The number of ENIs that Firehose creates in the subnets specified here scales up and down automatically based on throughput. To enable Firehose to scale up the number of ENIs to match throughput, ensure that you have sufficient quota. To help you calculate the quota you need, assume that Firehose can create up to three ENIs for this delivery stream for each of the subnets specified here. For more information about ENI quota, see Network Interfaces in the Amazon VPC Quotas topic.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300 (5 minutes).
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5.We recommend setting this parameter to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec, the value should be 10 MB or higher.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
Enables or disables CloudWatch logging.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
The CloudWatch group name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
The CloudWatch log stream name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_ClusterEndpoint <String>
The endpoint to use when communicating with the cluster. Specify either this ClusterEndpoint or the DomainARN field.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_DocumentIdOptions_DefaultDocumentIdFormat <DefaultDocumentIdFormat>
When the FIREHOSE_DEFAULT option is chosen, Firehose generates a unique document ID for each record based on a unique internal identifier. The generated document ID is stable across multiple delivery attempts, which helps prevent the same record from being indexed multiple times with different document IDs.When the NO_DOCUMENT_ID option is chosen, Firehose does not include any document IDs in the requests it sends to the Amazon OpenSearch Service. This causes the Amazon OpenSearch Service domain to generate document IDs. In case of multiple delivery attempts, this may cause the same record to be indexed more than once with different document IDs. This option enables write-heavy operations, such as the ingestion of logs and observability data, to consume less resources in the Amazon OpenSearch Service domain, resulting in improved performance.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_DomainARN <String>
The ARN of the Amazon OpenSearch Service domain. The IAM role must have permissions for DescribeElasticsearchDomain, DescribeElasticsearchDomains, and DescribeElasticsearchDomainConfig after assuming the role specified in RoleARN.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_IndexName <String>
The ElasticsearAmazon OpenSearch Service index name.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_IndexRotationPeriod <AmazonopensearchserviceIndexRotationPeriod>
The Amazon OpenSearch Service index rotation period. Index rotation appends a timestamp to the IndexName to facilitate the expiration of old data.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
Enables or disables data processing.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
The data processors.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
After an initial failure to deliver to Amazon OpenSearch Service, the total amount of time during which Firehose retries delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. Default value is 300 seconds (5 minutes). A value of 0 (zero) results in no retries.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_RoleARN <String>
The Amazon Resource Name (ARN) of the IAM role to be assumed by Firehose for calling the Amazon OpenSearch Service Configuration API and for indexing documents.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_S3BackupMode <AmazonopensearchserviceS3BackupMode>
Defines how documents should be delivered to Amazon S3. When it is set to FailedDocumentsOnly, Firehose writes any documents that could not be indexed to the configured Amazon S3 destination, with AmazonOpenSearchService-failed/ appended to the key prefix. When set to AllDocuments, Firehose delivers all incoming records to Amazon S3, and also writes failed documents with AmazonOpenSearchService-failed/ appended to the prefix.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
The service has not provided documentation for this parameter; please refer to the service's API reference documentation for the latest available information.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_TypeName <String>
The Amazon OpenSearch Service type name. For Elasticsearch 6.x, there can be only one type per index. If you try to specify a new type for an existing index that already has another type, Firehose returns an error during run time.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_RoleARN <String>
The ARN of the IAM role that you want the delivery stream to use to create endpoints in the destination VPC. You can use your existing Firehose delivery role or you can specify a new role. In either case, make sure that the role trusts the Firehose service principal and that it grants the following permissions:
  • ec2:DescribeVpcs
  • ec2:DescribeVpcAttribute
  • ec2:DescribeSubnets
  • ec2:DescribeSecurityGroups
  • ec2:DescribeNetworkInterfaces
  • ec2:CreateNetworkInterface
  • ec2:CreateNetworkInterfacePermission
  • ec2:DeleteNetworkInterface
When you specify subnets for delivering data to the destination in a private VPC, make sure you have enough number of free IP addresses in chosen subnets. If there is no available free IP address in a specified subnet, Firehose cannot create or add ENIs for the data delivery in the private VPC, and the delivery will be degraded or fail.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_SecurityGroupIds <String[]>
The IDs of the security groups that you want Firehose to use when it creates ENIs in the VPC of the Amazon ES destination. You can use the same security group that the Amazon ES domain uses or different ones. If you specify different security groups here, ensure that they allow outbound HTTPS traffic to the Amazon ES domain's security group. Also ensure that the Amazon ES domain's security group allows HTTPS traffic from the security groups specified here. If you use the same security group for both your delivery stream and the Amazon ES domain, make sure the security group inbound rule allows HTTPS traffic. For more information about security group rules, see Security group rules in the Amazon VPC documentation.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AmazonopensearchserviceDestinationConfiguration_VpcConfiguration_SubnetIds <String[]>
The IDs of the subnets that you want Firehose to use to create ENIs in the VPC of the Amazon ES destination. Make sure that the routing tables and inbound and outbound rules allow traffic to flow from the subnets whose IDs are specified here to the subnets that have the destination Amazon ES endpoints. Firehose creates at least one ENI in each of the subnets that are specified here. Do not delete or modify these ENIs.The number of ENIs that Firehose creates in the subnets specified here scales up and down automatically based on throughput. To enable Firehose to scale up the number of ENIs to match throughput, ensure that you have sufficient quota. To help you calculate the quota you need, assume that Firehose can create up to three ENIs for this delivery stream for each of the subnets specified here. For more information about ENI quota, see Network Interfaces in the Amazon VPC Quotas topic.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-AuthenticationConfiguration_Connectivity <Connectivity>
The type of connectivity used to access the Amazon MSK cluster.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesMSKSourceConfiguration_AuthenticationConfiguration_Connectivity
-AuthenticationConfiguration_RoleARN <String>
The ARN of the role used to access the Amazon MSK cluster.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesMSKSourceConfiguration_AuthenticationConfiguration_RoleARN
-BufferingHints_IntervalInSecond <Int32>
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300 (5 minutes).
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_BufferingHints_IntervalInSeconds
-BufferingHints_SizeInMBs <Int32>
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5.We recommend setting this parameter to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec, the value should be 10 MB or higher.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_BufferingHints_SizeInMBs
-CatalogConfiguration_CatalogARN <String>
Specifies the Glue catalog ARN indentifier of the destination Apache Iceberg Tables. You must specify the ARN in the format arn:aws:glue:region:account-id:catalog.Amazon Data Firehose is in preview release and is subject to change.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesIcebergDestinationConfiguration_CatalogConfiguration_CatalogARN
-ClientConfig <AmazonKinesisFirehoseConfig>
Amazon.PowerShell.Cmdlets.KINF.AmazonKinesisFirehoseClientCmdlet.ClientConfig
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-CloudWatchLoggingOptions_Enabled <Boolean>
Enables or disables CloudWatch logging.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_CloudWatchLoggingOptions_Enabled
-CloudWatchLoggingOptions_LogGroupName <String>
The CloudWatch group name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName
-CloudWatchLoggingOptions_LogStreamName <String>
The CloudWatch log stream name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName
-DeliveryStreamEncryptionConfigurationInput_KeyARN <String>
If you set KeyType to CUSTOMER_MANAGED_CMK, you must specify the Amazon Resource Name (ARN) of the CMK. If you set KeyType to Amazon Web Services_OWNED_CMK, Firehose uses a service-account CMK.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DeliveryStreamEncryptionConfigurationInput_KeyType <KeyType>
Indicates the type of customer master key (CMK) to use for encryption. The default setting is Amazon Web Services_OWNED_CMK. For more information about CMKs, see Customer Master Keys (CMKs). When you invoke CreateDeliveryStream or StartDeliveryStreamEncryption with KeyType set to CUSTOMER_MANAGED_CMK, Firehose invokes the Amazon KMS operation CreateGrant to create a grant that allows the Firehose service to use the customer managed CMK to perform encryption and decryption. Firehose manages that grant. When you invoke StartDeliveryStreamEncryption to change the CMK for a delivery stream that is encrypted with a customer managed CMK, Firehose schedules the grant it had on the old CMK for retirement.You can use a CMK of type CUSTOMER_MANAGED_CMK to encrypt up to 500 delivery streams. If a CreateDeliveryStream or StartDeliveryStreamEncryption operation exceeds this limit, Firehose throws a LimitExceededException. To encrypt your delivery stream, use symmetric CMKs. Firehose doesn't support asymmetric CMKs. For information about symmetric and asymmetric CMKs, see About Symmetric and Asymmetric CMKs in the Amazon Web Services Key Management Service developer guide.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DeliveryStreamName <String>
The name of the delivery stream. This name must be unique per Amazon Web Services account in the same Amazon Web Services Region. If the delivery streams are in different accounts or different Regions, you can have multiple delivery streams with the same name.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)
-DeliveryStreamType <DeliveryStreamType>
The delivery stream type. This parameter can be one of the following values:
  • DirectPut: Provider applications access the delivery stream directly.
  • KinesisStreamAsSource: The delivery stream uses a Kinesis data stream as a source.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DocumentIdOptions_DefaultDocumentIdFormat <DefaultDocumentIdFormat>
When the FIREHOSE_DEFAULT option is chosen, Firehose generates a unique document ID for each record based on a unique internal identifier. The generated document ID is stable across multiple delivery attempts, which helps prevent the same record from being indexed multiple times with different document IDs.When the NO_DOCUMENT_ID option is chosen, Firehose does not include any document IDs in the requests it sends to the Amazon OpenSearch Service. This causes the Amazon OpenSearch Service domain to generate document IDs. In case of multiple delivery attempts, this may cause the same record to be indexed more than once with different document IDs. This option enables write-heavy operations, such as the ingestion of logs and observability data, to consume less resources in the Amazon OpenSearch Service domain, resulting in improved performance.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_DocumentIdOptions_DefaultDocumentIdFormat
-ElasticsearchDestinationConfiguration_ClusterEndpoint <String>
The endpoint to use when communicating with the cluster. Specify either this ClusterEndpoint or the DomainARN field.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_DomainARN <String>
The ARN of the Amazon ES domain. The IAM role must have permissions for DescribeDomain, DescribeDomains, and DescribeDomainConfig after assuming the role specified in RoleARN. For more information, see Amazon Resource Names (ARNs) and Amazon Web Services Service Namespaces.Specify either ClusterEndpoint or DomainARN.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_IndexName <String>
The Elasticsearch index name.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_IndexRotationPeriod <ElasticsearchIndexRotationPeriod>
The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate the expiration of old data. For more information, see Index Rotation for the Amazon ES Destination. The default value is OneDay.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_RoleARN <String>
The Amazon Resource Name (ARN) of the IAM role to be assumed by Firehose for calling the Amazon ES Configuration API and for indexing documents. For more information, see Grant Firehose Access to an Amazon S3 Destination and Amazon Resource Names (ARNs) and Amazon Web Services Service Namespaces.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_S3BackupMode <ElasticsearchS3BackupMode>
Defines how documents should be delivered to Amazon S3. When it is set to FailedDocumentsOnly, Firehose writes any documents that could not be indexed to the configured Amazon S3 destination, with AmazonOpenSearchService-failed/ appended to the key prefix. When set to AllDocuments, Firehose delivers all incoming records to Amazon S3, and also writes failed documents with AmazonOpenSearchService-failed/ appended to the prefix. For more information, see Amazon S3 Backup for the Amazon ES Destination. Default value is FailedDocumentsOnly.You can't change this backup mode after you create the delivery stream.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
The configuration for the backup Amazon S3 location.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ElasticsearchDestinationConfiguration_TypeName <String>
The Elasticsearch type name. For Elasticsearch 6.x, there can be only one type per index. If you try to specify a new type for an existing index that already has another type, Firehose returns an error during run time.For Elasticsearch 7.x, don't specify a TypeName.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-EndpointConfiguration_Name <String>
The name of the HTTP endpoint selected as the destination.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesHttpEndpointDestinationConfiguration_EndpointConfiguration_Name
-ExtendedS3DestinationConfiguration <ExtendedS3DestinationConfiguration>
The destination in Amazon S3. You can specify only one destination.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-Force <SwitchParameter>
This parameter overrides confirmation prompts to force the cmdlet to continue its operation. This parameter should always be used with caution.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300 (5 minutes).
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting this parameter to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec, the value should be 10 MB or higher.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
Enables or disables CloudWatch logging.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
The CloudWatch group name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
The CloudWatch log stream name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_EndpointConfiguration_AccessKey <String>
The access key required for Kinesis Firehose to authenticate with the HTTP endpoint selected as the destination.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_EndpointConfiguration_Url <String>
The URL of the HTTP endpoint selected as the destination.If you choose an HTTP endpoint as your destination, review and follow the instructions in the Appendix - HTTP Endpoint Delivery Request and Response Specifications.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
Enables or disables data processing.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
The data processors.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_RequestConfiguration_CommonAttributes <HttpEndpointCommonAttribute[]>
Describes the metadata sent to the HTTP endpoint destination.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_RequestConfiguration_ContentEncoding <ContentEncoding>
Firehose uses the content encoding to compress the body of a request before sending the request to the destination. For more information, see Content-Encoding in MDN Web Docs, the official Mozilla documentation.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
The total amount of time that Firehose spends on retries. This duration starts after the initial attempt to send data to the custom destination via HTTPS endpoint fails. It doesn't include the periods during which Firehose waits for acknowledgment from the specified destination after each attempt.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_RoleARN <String>
Firehose uses this IAM role for all the permissions that the delivery stream needs.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_S3BackupMode <HttpEndpointS3BackupMode>
Describes the S3 bucket backup options for the data that Firehose delivers to the HTTP endpoint destination. You can back up all documents (AllData) or only the documents that Firehose could not deliver to the specified HTTP endpoint destination (FailedDataOnly).
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
The service has not provided documentation for this parameter; please refer to the service's API reference documentation for the latest available information.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_SecretsManagerConfiguration_Enabled <Boolean>
Specifies whether you want to use the the secrets manager feature. When set as True the secrets manager configuration overwrites the existing secrets in the destination configuration. When it's set to False Firehose falls back to the credentials in the destination configuration.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_SecretsManagerConfiguration_RoleARN <String>
Specifies the role that Firehose assumes when calling the Secrets Manager API operation. When you provide the role, it overrides any destination specific role defined in the destination configuration. If you do not provide the then we use the destination specific role. This parameter is required for Splunk.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-HttpEndpointDestinationConfiguration_SecretsManagerConfiguration_SecretARN <String>
The ARN of the secret that stores your credentials. It must be in the same region as the Firehose stream and the role. The secret ARN can reside in a different account than the delivery stream and role as Firehose supports cross-account secret access. This parameter is required when Enabled is set to True.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 300. This parameter is optional but if you specify a value for it, you must also specify a value for SizeInMBs, and vice versa.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
Buffer incoming data to the specified size, in MiBs, before delivering it to the destination. The default value is 5. This parameter is optional but if you specify a value for it, you must also specify a value for IntervalInSeconds, and vice versa.We recommend setting this parameter to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MiB/sec, the value should be 10 MiB or higher.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
Enables or disables CloudWatch logging.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
The CloudWatch group name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
The CloudWatch log stream name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_DestinationTableConfigurationList <DestinationTableConfiguration[]>
Provides a list of DestinationTableConfigurations which Firehose uses to deliver data to Apache Iceberg tables. Amazon Data Firehose is in preview release and is subject to change.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
Enables or disables data processing.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
The data processors.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
The period of time during which Firehose retries to deliver data to the specified Amazon S3 prefix.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_RoleARN <String>
The Amazon Resource Name (ARN) of the Apache Iceberg tables role. Amazon Data Firehose is in preview release and is subject to change.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_S3BackupMode <IcebergS3BackupMode>
Describes how Firehose will backup records. Currently,Firehose only supports FailedDataOnly for preview. Amazon Data Firehose is in preview release and is subject to change.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-IcebergDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
The service has not provided documentation for this parameter; please refer to the service's API reference documentation for the latest available information.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-KinesisStreamSourceConfiguration_KinesisStreamARN <String>
The ARN of the source Kinesis data stream. For more information, see Amazon Kinesis Data Streams ARN Format.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-KinesisStreamSourceConfiguration_RoleARN <String>
The ARN of the role that provides access to the source Kinesis data stream. For more information, see Amazon Web Services Identity and Access Management (IAM) ARN Format.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-MSKSourceConfiguration_MSKClusterARN <String>
The ARN of the Amazon MSK cluster.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-MSKSourceConfiguration_ReadFromTimestamp <DateTime>
The start date and time in UTC for the offset position within your MSK topic from where Firehose begins to read. By default, this is set to timestamp when Firehose becomes Active. If you want to create a Firehose stream with Earliest start position from SDK or CLI, you need to set the ReadFromTimestamp parameter to Epoch (1970-01-01T00:00:00Z).
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-MSKSourceConfiguration_TopicName <String>
The topic name within the Amazon MSK cluster.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ProcessingConfiguration_Enabled <Boolean>
Enables or disables data processing.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_ProcessingConfiguration_Enabled
-ProcessingConfiguration_Processor <Processor[]>
The data processors.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_ProcessingConfiguration_Processors
-RedshiftDestinationConfiguration <RedshiftDestinationConfiguration>
The destination in Amazon Redshift. You can specify only one destination.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-RetryOptions_DurationInSecond <Int32>
After an initial failure to deliver to Amazon ES, the total amount of time during which Firehose retries delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. Default value is 300 seconds (5 minutes). A value of 0 (zero) results in no retries.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_RetryOptions_DurationInSeconds
-S3DestinationConfiguration <S3DestinationConfiguration>
[Deprecated] The destination in Amazon S3. You can specify only one destination.This parameter is deprecated.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-Select <String>
Use the -Select parameter to control the cmdlet output. The default value is 'DeliveryStreamARN'. Specifying -Select '*' will result in the cmdlet returning the whole service response (Amazon.KinesisFirehose.Model.CreateDeliveryStreamResponse). Specifying the name of a property of type Amazon.KinesisFirehose.Model.CreateDeliveryStreamResponse will result in that property being returned. Specifying -Select '^ParameterName' will result in the cmdlet returning the selected cmdlet parameter value.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_AccountUrl <String>
URL for accessing your Snowflake account. This URL must include your account identifier. Note that the protocol (https://) and port number are optional.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_BufferingHints_IntervalInSeconds <Int32>
Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. The default value is 0.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_BufferingHints_SizeInMBs <Int32>
Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 1.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_CloudWatchLoggingOptions_Enabled <Boolean>
Enables or disables CloudWatch logging.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_CloudWatchLoggingOptions_LogGroupName <String>
The CloudWatch group name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_CloudWatchLoggingOptions_LogStreamName <String>
The CloudWatch log stream name for logging. This value is required if CloudWatch logging is enabled.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_ContentColumnName <String>
The name of the record content column
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_Database <String>
All data in Snowflake is maintained in databases.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_DataLoadingOption <SnowflakeDataLoadingOption>
Choose to load JSON keys mapped to table column names or choose to split the JSON payload where content is mapped to a record content column and source metadata is mapped to a record metadata column.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_KeyPassphrase <String>
Passphrase to decrypt the private key when the key is encrypted. For information, see Using Key Pair Authentication & Key Rotation.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_MetaDataColumnName <String>
The name of the record metadata column
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_PrivateKey <String>
The private key used to encrypt your Snowflake client. For information, see Using Key Pair Authentication & Key Rotation.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_ProcessingConfiguration_Enabled <Boolean>
Enables or disables data processing.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_ProcessingConfiguration_Processors <Processor[]>
The data processors.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_RetryOptions_DurationInSeconds <Int32>
the time period where Firehose will retry sending data to the chosen HTTP endpoint.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_RoleARN <String>
The Amazon Resource Name (ARN) of the Snowflake role
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_S3BackupMode <SnowflakeS3BackupMode>
Choose an S3 backup mode
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_S3Configuration <S3DestinationConfiguration>
The service has not provided documentation for this parameter; please refer to the service's API reference documentation for the latest available information.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_Schema <String>
Each database consists of one or more schemas, which are logical groupings of database objects, such as tables and views
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_SecretsManagerConfiguration_Enabled <Boolean>
Specifies whether you want to use the the secrets manager feature. When set as True the secrets manager configuration overwrites the existing secrets in the destination configuration. When it's set to False Firehose falls back to the credentials in the destination configuration.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_SecretsManagerConfiguration_RoleARN <String>
Specifies the role that Firehose assumes when calling the Secrets Manager API operation. When you provide the role, it overrides any destination specific role defined in the destination configuration. If you do not provide the then we use the destination specific role. This parameter is required for Splunk.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_SecretsManagerConfiguration_SecretARN <String>
The ARN of the secret that stores your credentials. It must be in the same region as the Firehose stream and the role. The secret ARN can reside in a different account than the delivery stream and role as Firehose supports cross-account secret access. This parameter is required when Enabled is set to True.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_Table <String>
All data in Snowflake is stored in database tables, logically structured as collections of columns and rows.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeDestinationConfiguration_User <String>
User login name for the Snowflake account.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SnowflakeRoleConfiguration_Enabled <Boolean>
Enable Snowflake role
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesSnowflakeDestinationConfiguration_SnowflakeRoleConfiguration_Enabled
-SnowflakeRoleConfiguration_SnowflakeRole <String>
The Snowflake role you wish to configure
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesSnowflakeDestinationConfiguration_SnowflakeRoleConfiguration_SnowflakeRole
-SnowflakeVpcConfiguration_PrivateLinkVpceId <String>
The VPCE ID for Firehose to privately connect with Snowflake. The ID format is com.amazonaws.vpce.[region].vpce-svc-<[id]>. For more information, see Amazon PrivateLink & Snowflake
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesSnowflakeDestinationConfiguration_SnowflakeVpcConfiguration_PrivateLinkVpceId
-SplunkDestinationConfiguration <SplunkDestinationConfiguration>
The destination in Splunk. You can specify only one destination.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-Tag <Tag[]>
A set of tags to assign to the delivery stream. A tag is a key-value pair that you can define and assign to Amazon Web Services resources. Tags are metadata. For example, you can add friendly names and descriptions or other types of information that can help you distinguish the delivery stream. For more information about tags, see Using Cost Allocation Tags in the Amazon Web Services Billing and Cost Management User Guide.You can specify up to 50 tags when creating a delivery stream.If you specify tags in the CreateDeliveryStream action, Amazon Data Firehose performs an additional authorization on the firehose:TagDeliveryStream action to verify if users have permissions to create tags. If you do not provide this permission, requests to create new Firehose delivery streams with IAM resource tags will fail with an AccessDeniedException such as following.AccessDeniedExceptionUser: arn:aws:sts::x:assumed-role/x/x is not authorized to perform: firehose:TagDeliveryStream on resource: arn:aws:firehose:us-east-1:x:deliverystream/x with an explicit deny in an identity-based policy.For an example IAM policy, see Tag example.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesTags
-VpcConfiguration_RoleARN <String>
The ARN of the IAM role that you want the delivery stream to use to create endpoints in the destination VPC. You can use your existing Firehose delivery role or you can specify a new role. In either case, make sure that the role trusts the Firehose service principal and that it grants the following permissions:
  • ec2:DescribeVpcs
  • ec2:DescribeVpcAttribute
  • ec2:DescribeSubnets
  • ec2:DescribeSecurityGroups
  • ec2:DescribeNetworkInterfaces
  • ec2:CreateNetworkInterface
  • ec2:CreateNetworkInterfacePermission
  • ec2:DeleteNetworkInterface
When you specify subnets for delivering data to the destination in a private VPC, make sure you have enough number of free IP addresses in chosen subnets. If there is no available free IP address in a specified subnet, Firehose cannot create or add ENIs for the data delivery in the private VPC, and the delivery will be degraded or fail.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_VpcConfiguration_RoleARN
-VpcConfiguration_SecurityGroupId <String[]>
The IDs of the security groups that you want Firehose to use when it creates ENIs in the VPC of the Amazon ES destination. You can use the same security group that the Amazon ES domain uses or different ones. If you specify different security groups here, ensure that they allow outbound HTTPS traffic to the Amazon ES domain's security group. Also ensure that the Amazon ES domain's security group allows HTTPS traffic from the security groups specified here. If you use the same security group for both your delivery stream and the Amazon ES domain, make sure the security group inbound rule allows HTTPS traffic. For more information about security group rules, see Security group rules in the Amazon VPC documentation.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_VpcConfiguration_SecurityGroupIds
-VpcConfiguration_SubnetId <String[]>
The IDs of the subnets that you want Firehose to use to create ENIs in the VPC of the Amazon ES destination. Make sure that the routing tables and inbound and outbound rules allow traffic to flow from the subnets whose IDs are specified here to the subnets that have the destination Amazon ES endpoints. Firehose creates at least one ENI in each of the subnets that are specified here. Do not delete or modify these ENIs.The number of ENIs that Firehose creates in the subnets specified here scales up and down automatically based on throughput. To enable Firehose to scale up the number of ENIs to match throughput, ensure that you have sufficient quota. To help you calculate the quota you need, assume that Firehose can create up to three ENIs for this delivery stream for each of the subnets specified here. For more information about ENI quota, see Network Interfaces in the Amazon VPC Quotas topic.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesElasticsearchDestinationConfiguration_VpcConfiguration_SubnetIds

Common Credential and Region Parameters

-AccessKey <String>
The AWS access key for the user account. This can be a temporary access key if the corresponding session token is supplied to the -SessionToken parameter.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesAK
-Credential <AWSCredentials>
An AWSCredentials object instance containing access and secret key information, and optionally a token for session-based credentials.
Required?False
Position?Named
Accept pipeline input?True (ByValue, ByPropertyName)
-EndpointUrl <String>
The endpoint to make the call against.Note: This parameter is primarily for internal AWS use and is not required/should not be specified for normal usage. The cmdlets normally determine which endpoint to call based on the region specified to the -Region parameter or set as default in the shell (via Set-DefaultAWSRegion). Only specify this parameter if you must direct the call to a specific custom endpoint.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-NetworkCredential <PSCredential>
Used with SAML-based authentication when ProfileName references a SAML role profile. Contains the network credentials to be supplied during authentication with the configured identity provider's endpoint. This parameter is not required if the user's default network identity can or should be used during authentication.
Required?False
Position?Named
Accept pipeline input?True (ByValue, ByPropertyName)
-ProfileLocation <String>
Used to specify the name and location of the ini-format credential file (shared with the AWS CLI and other AWS SDKs)If this optional parameter is omitted this cmdlet will search the encrypted credential file used by the AWS SDK for .NET and AWS Toolkit for Visual Studio first. If the profile is not found then the cmdlet will search in the ini-format credential file at the default location: (user's home directory)\.aws\credentials.If this parameter is specified then this cmdlet will only search the ini-format credential file at the location given.As the current folder can vary in a shell or during script execution it is advised that you use specify a fully qualified path instead of a relative path.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesAWSProfilesLocation, ProfilesLocation
-ProfileName <String>
The user-defined name of an AWS credentials or SAML-based role profile containing credential information. The profile is expected to be found in the secure credential file shared with the AWS SDK for .NET and AWS Toolkit for Visual Studio. You can also specify the name of a profile stored in the .ini-format credential file used with the AWS CLI and other AWS SDKs.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesStoredCredentials, AWSProfileName
-Region <Object>
The system name of an AWS region or an AWSRegion instance. This governs the endpoint that will be used when calling service operations. Note that the AWS resources referenced in a call are usually region-specific.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesRegionToCall
-SecretKey <String>
The AWS secret key for the user account. This can be a temporary secret key if the corresponding session token is supplied to the -SessionToken parameter.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesSK, SecretAccessKey
-SessionToken <String>
The session token if the access and secret keys are temporary session-based credentials.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesST

Outputs

System.String or Amazon.KinesisFirehose.Model.CreateDeliveryStreamResponse
This cmdlet returns a System.String object. The service call response (type Amazon.KinesisFirehose.Model.CreateDeliveryStreamResponse) can also be referenced from properties attached to the cmdlet entry in the $AWSHistory stack.
Developer Guide
Service API Reference
AWS Tools for PowerShell User Guide

Supported Version

AWS Tools for PowerShell: 2.x.y.z