UpdateDestination - Amazon Data Firehose

UpdateDestination

Updates the specified destination of the specified Firehose stream.

Use this operation to change the destination type (for example, to replace the Amazon S3 destination with Amazon Redshift) or change the parameters associated with a destination (for example, to change the bucket name of the Amazon S3 destination). The update might not occur immediately. The target Firehose stream remains active while the configurations are updated, so data writes to the Firehose stream can continue during this process. The updated configurations are usually effective within a few minutes.

Switching between Amazon OpenSearch Service and other services is not supported. For an Amazon OpenSearch Service destination, you can only update to another Amazon OpenSearch Service destination.

If the destination type is the same, Firehose merges the configuration parameters specified with the destination configuration that already exists on the delivery stream. If any of the parameters are not specified in the call, the existing values are retained. For example, in the Amazon S3 destination, if EncryptionConfiguration is not specified, then the existing EncryptionConfiguration is maintained on the destination.

If the destination type is not the same, for example, changing the destination from Amazon S3 to Amazon Redshift, Firehose does not merge any parameters. In this case, all parameters must be specified.

Firehose uses CurrentDeliveryStreamVersionId to avoid race conditions and conflicting merges. This is a required field, and the service updates the configuration only if the existing configuration has a version ID that matches. After the update is applied successfully, the version ID is updated, and can be retrieved using DescribeDeliveryStream. Use the new version ID to set CurrentDeliveryStreamVersionId in the next call.

Request Syntax

{ "AmazonOpenSearchServerlessDestinationUpdate": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CollectionEndpoint": "string", "IndexName": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" } }, "AmazonopensearchserviceDestinationUpdate": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "ClusterEndpoint": "string", "DocumentIdOptions": { "DefaultDocumentIdFormat": "string" }, "DomainARN": "string", "IndexName": "string", "IndexRotationPeriod": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "TypeName": "string" }, "CurrentDeliveryStreamVersionId": "string", "DeliveryStreamName": "string", "DestinationId": "string", "ElasticsearchDestinationUpdate": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "ClusterEndpoint": "string", "DocumentIdOptions": { "DefaultDocumentIdFormat": "string" }, "DomainARN": "string", "IndexName": "string", "IndexRotationPeriod": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "TypeName": "string" }, "ExtendedS3DestinationUpdate": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "CustomTimeZone": "string", "DataFormatConversionConfiguration": { "Enabled": boolean, "InputFormatConfiguration": { "Deserializer": { "HiveJsonSerDe": { "TimestampFormats": [ "string" ] }, "OpenXJsonSerDe": { "CaseInsensitive": boolean, "ColumnToJsonKeyMappings": { "string" : "string" }, "ConvertDotsInJsonKeysToUnderscores": boolean } } }, "OutputFormatConfiguration": { "Serializer": { "OrcSerDe": { "BlockSizeBytes": number, "BloomFilterColumns": [ "string" ], "BloomFilterFalsePositiveProbability": number, "Compression": "string", "DictionaryKeyThreshold": number, "EnablePadding": boolean, "FormatVersion": "string", "PaddingTolerance": number, "RowIndexStride": number, "StripeSizeBytes": number }, "ParquetSerDe": { "BlockSizeBytes": number, "Compression": "string", "EnableDictionaryCompression": boolean, "MaxPaddingBytes": number, "PageSizeBytes": number, "WriterVersion": "string" } } }, "SchemaConfiguration": { "CatalogId": "string", "DatabaseName": "string", "Region": "string", "RoleARN": "string", "TableName": "string", "VersionId": "string" } }, "DynamicPartitioningConfiguration": { "Enabled": boolean, "RetryOptions": { "DurationInSeconds": number } }, "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "FileExtension": "string", "Prefix": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RoleARN": "string", "S3BackupMode": "string", "S3BackupUpdate": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" } }, "HttpEndpointDestinationUpdate": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "EndpointConfiguration": { "AccessKey": "string", "Name": "string", "Url": "string" }, "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RequestConfiguration": { "CommonAttributes": [ { "AttributeName": "string", "AttributeValue": "string" } ], "ContentEncoding": "string" }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3BackupMode": "string", "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "SecretsManagerConfiguration": { "Enabled": boolean, "RoleARN": "string", "SecretARN": "string" } }, "IcebergDestinationUpdate": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CatalogConfiguration": { "CatalogARN": "string" }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "DestinationTableConfigurationList": [ { "DestinationDatabaseName": "string", "DestinationTableName": "string", "S3ErrorOutputPrefix": "string", "UniqueKeys": [ "string" ] } ], "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3BackupMode": "string", "S3Configuration": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" } }, "RedshiftDestinationUpdate": { "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "ClusterJDBCURL": "string", "CopyCommand": { "CopyOptions": "string", "DataTableColumns": "string", "DataTableName": "string" }, "Password": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3BackupMode": "string", "S3BackupUpdate": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "SecretsManagerConfiguration": { "Enabled": boolean, "RoleARN": "string", "SecretARN": "string" }, "Username": "string" }, "S3DestinationUpdate": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "SnowflakeDestinationUpdate": { "AccountUrl": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "ContentColumnName": "string", "Database": "string", "DataLoadingOption": "string", "KeyPassphrase": "string", "MetaDataColumnName": "string", "PrivateKey": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "RoleARN": "string", "S3BackupMode": "string", "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "Schema": "string", "SecretsManagerConfiguration": { "Enabled": boolean, "RoleARN": "string", "SecretARN": "string" }, "SnowflakeRoleConfiguration": { "Enabled": boolean, "SnowflakeRole": "string" }, "Table": "string", "User": "string" }, "SplunkDestinationUpdate": { "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "HECAcknowledgmentTimeoutInSeconds": number, "HECEndpoint": "string", "HECEndpointType": "string", "HECToken": "string", "ProcessingConfiguration": { "Enabled": boolean, "Processors": [ { "Parameters": [ { "ParameterName": "string", "ParameterValue": "string" } ], "Type": "string" } ] }, "RetryOptions": { "DurationInSeconds": number }, "S3BackupMode": "string", "S3Update": { "BucketARN": "string", "BufferingHints": { "IntervalInSeconds": number, "SizeInMBs": number }, "CloudWatchLoggingOptions": { "Enabled": boolean, "LogGroupName": "string", "LogStreamName": "string" }, "CompressionFormat": "string", "EncryptionConfiguration": { "KMSEncryptionConfig": { "AWSKMSKeyARN": "string" }, "NoEncryptionConfig": "string" }, "ErrorOutputPrefix": "string", "Prefix": "string", "RoleARN": "string" }, "SecretsManagerConfiguration": { "Enabled": boolean, "RoleARN": "string", "SecretARN": "string" } } }

Request Parameters

The request accepts the following data in JSON format.

AmazonOpenSearchServerlessDestinationUpdate

Describes an update for a destination in the Serverless offering for Amazon OpenSearch Service.

Type: AmazonOpenSearchServerlessDestinationUpdate object

Required: No

AmazonopensearchserviceDestinationUpdate

Describes an update for a destination in Amazon OpenSearch Service.

Type: AmazonopensearchserviceDestinationUpdate object

Required: No

CurrentDeliveryStreamVersionId

Obtain this value from the VersionId result of DeliveryStreamDescription. This value is required, and helps the service perform conditional operations. For example, if there is an interleaving update and this value is null, then the update destination fails. After the update is successful, the VersionId value is updated. The service then performs a merge of the old configuration with the new configuration.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 50.

Pattern: [0-9]+

Required: Yes

DeliveryStreamName

The name of the Firehose stream.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 64.

Pattern: [a-zA-Z0-9_.-]+

Required: Yes

DestinationId

The ID of the destination.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 100.

Pattern: [a-zA-Z0-9-]+

Required: Yes

ElasticsearchDestinationUpdate

Describes an update for a destination in Amazon OpenSearch Service.

Type: ElasticsearchDestinationUpdate object

Required: No

ExtendedS3DestinationUpdate

Describes an update for a destination in Amazon S3.

Type: ExtendedS3DestinationUpdate object

Required: No

HttpEndpointDestinationUpdate

Describes an update to the specified HTTP endpoint destination.

Type: HttpEndpointDestinationUpdate object

Required: No

IcebergDestinationUpdate

Describes an update for a destination in Apache Iceberg Tables.

Type: IcebergDestinationUpdate object

Required: No

RedshiftDestinationUpdate

Describes an update for a destination in Amazon Redshift.

Type: RedshiftDestinationUpdate object

Required: No

S3DestinationUpdate

[Deprecated] Describes an update for a destination in Amazon S3.

Type: S3DestinationUpdate object

Required: No

SnowflakeDestinationUpdate

Update to the Snowflake destination configuration settings.

Type: SnowflakeDestinationUpdate object

Required: No

SplunkDestinationUpdate

Describes an update for a destination in Splunk.

Type: SplunkDestinationUpdate object

Required: No

Response Elements

If the action is successful, the service sends back an HTTP 200 response with an empty HTTP body.

Errors

For information about the errors that are common to all actions, see Common Errors.

ConcurrentModificationException

Another modification has already happened. Fetch VersionId again and use it to update the destination.

HTTP Status Code: 400

InvalidArgumentException

The specified input parameter has a value that is not valid.

HTTP Status Code: 400

ResourceInUseException

The resource is already in use and not available for this operation.

HTTP Status Code: 400

ResourceNotFoundException

The specified resource could not be found.

HTTP Status Code: 400

Examples

Example

The following JSON updates the buffering hints for the destination to 150 seconds and 2 MiB. You can obtain the other values that you need to use in this example from the response to a DescribeDeliveryStream invocation for the Firehose stream in question.

Sample Request

POST / HTTP/1.1 Host: firehose.<region>.<domain> Content-Length: <PayloadSizeBytes> User-Agent: <UserAgentString> Content-Type: application/x-amz-json-1.1 Authorization: <AuthParams> Connection: Keep-Alive X-Amz-Date: <Date> X-Amz-Target: Firehose_20150804.UpdateDestination { "CurrentDeliveryStreamVersionId": "1", "DeliveryStreamName": "exampleStreamName", "DestinationId": "destinationId-000000000001", "ExtendedS3DestinationUpdate": { "BucketARN": "arn:aws:s3:::somebucket", "BufferingHints": { "IntervalInSeconds": 150, "SizeInMBs": 2 }, "RoleARN": "arn:aws:iam::111222333444:role/exampleStreamName" } }

Sample Response

HTTP/1.1 200 OK x-amzn-RequestId: <RequestId> Content-Type: application/x-amz-json-1.1 Content-Length: <PayloadSizeBytes> Date: <Date>

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following: