Class: Aws::DatabaseMigrationService::Types::S3Settings
- Inherits:
-
Struct
- Object
- Struct
- Aws::DatabaseMigrationService::Types::S3Settings
- Defined in:
- gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb
Overview
Settings for exporting data to Amazon S3.
Constant Summary collapse
- SENSITIVE =
[]
Instance Attribute Summary collapse
-
#add_column_name ⇒ Boolean
An optional parameter that, when set to
true
ory
, you can use to add column name information to the .csv output file. -
#add_trailing_padding_character ⇒ Boolean
Use the S3 target endpoint setting
AddTrailingPaddingCharacter
to add padding on string data. -
#bucket_folder ⇒ String
An optional parameter to set a folder name in the S3 bucket.
-
#bucket_name ⇒ String
The name of the S3 bucket.
-
#canned_acl_for_objects ⇒ String
A value that enables DMS to specify a predefined (canned) access control list for objects created in an Amazon S3 bucket as .csv or .parquet files.
-
#cdc_inserts_and_updates ⇒ Boolean
A value that enables a change data capture (CDC) load to write INSERT and UPDATE operations to .csv or .parquet (columnar storage) output files.
-
#cdc_inserts_only ⇒ Boolean
A value that enables a change data capture (CDC) load to write only INSERT operations to .csv or columnar storage (.parquet) output files.
-
#cdc_max_batch_interval ⇒ Integer
Maximum length of the interval, defined in seconds, after which to output a file to Amazon S3.
-
#cdc_min_file_size ⇒ Integer
Minimum file size, defined in kilobytes, to reach for a file output to Amazon S3.
-
#cdc_path ⇒ String
Specifies the folder path of CDC files.
-
#compression_type ⇒ String
An optional parameter to use GZIP to compress the target files.
-
#csv_delimiter ⇒ String
The delimiter used to separate columns in the .csv file for both source and target.
-
#csv_no_sup_value ⇒ String
This setting only applies if your Amazon S3 output files during a change data capture (CDC) load are written in .csv format.
-
#csv_null_value ⇒ String
An optional parameter that specifies how DMS treats null values.
-
#csv_row_delimiter ⇒ String
The delimiter used to separate rows in the .csv file for both source and target.
-
#data_format ⇒ String
The format of the data that you want to use for output.
-
#data_page_size ⇒ Integer
The size of one data page in bytes.
-
#date_partition_delimiter ⇒ String
Specifies a date separating delimiter to use during folder partitioning.
-
#date_partition_enabled ⇒ Boolean
When set to
true
, this parameter partitions S3 bucket folders based on transaction commit dates. -
#date_partition_sequence ⇒ String
Identifies the sequence of the date format to use during folder partitioning.
-
#date_partition_timezone ⇒ String
When creating an S3 target endpoint, set
DatePartitionTimezone
to convert the current UTC time into a specified time zone. -
#dict_page_size_limit ⇒ Integer
The maximum size of an encoded dictionary page of a column.
-
#enable_statistics ⇒ Boolean
A value that enables statistics for Parquet pages and row groups.
-
#encoding_type ⇒ String
The type of encoding you are using:.
-
#encryption_mode ⇒ String
The type of server-side encryption that you want to use for your data.
-
#expected_bucket_owner ⇒ String
To specify a bucket owner and prevent sniping, you can use the
ExpectedBucketOwner
endpoint setting. -
#external_table_definition ⇒ String
Specifies how tables are defined in the S3 source files only.
-
#glue_catalog_generation ⇒ Boolean
When true, allows Glue to catalog your S3 bucket.
-
#ignore_header_rows ⇒ Integer
When this value is set to 1, DMS ignores the first row header in a .csv file.
-
#include_op_for_full_load ⇒ Boolean
A value that enables a full load to write INSERT operations to the comma-separated value (.csv) or .parquet output files only to indicate how the rows were added to the source database.
-
#max_file_size ⇒ Integer
A value that specifies the maximum size (in KB) of any .csv file to be created while migrating to an S3 target during full load.
-
#parquet_timestamp_in_millisecond ⇒ Boolean
A value that specifies the precision of any
TIMESTAMP
column values that are written to an Amazon S3 object file in .parquet format. -
#parquet_version ⇒ String
The version of the Apache Parquet format that you want to use:
parquet_1_0
(the default) orparquet_2_0
. -
#preserve_transactions ⇒ Boolean
If set to
true
, DMS saves the transaction order for a change data capture (CDC) load on the Amazon S3 target specified by [CdcPath
][1]. -
#rfc_4180 ⇒ Boolean
For an S3 source, when this value is set to
true
ory
, each leading double quotation mark has to be followed by an ending double quotation mark. -
#row_group_length ⇒ Integer
The number of rows in a row group.
-
#server_side_encryption_kms_key_id ⇒ String
If you are using
SSE_KMS
for theEncryptionMode
, provide the KMS key ID. -
#service_access_role_arn ⇒ String
The Amazon Resource Name (ARN) used by the service to access the IAM role.
-
#timestamp_column_name ⇒ String
A value that when nonblank causes DMS to add a column with timestamp information to the endpoint data for an Amazon S3 target.
-
#use_csv_no_sup_value ⇒ Boolean
This setting applies if the S3 output files during a change data capture (CDC) load are written in .csv format.
-
#use_task_start_time_for_full_load_timestamp ⇒ Boolean
When set to true, this parameter uses the task start time as the timestamp column value instead of the time data is written to target.
Instance Attribute Details
#add_column_name ⇒ Boolean
An optional parameter that, when set to true
or y
, you can use
to add column name information to the .csv output file.
The default value is false
. Valid values are true
, false
, y
,
and n
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#add_trailing_padding_character ⇒ Boolean
Use the S3 target endpoint setting AddTrailingPaddingCharacter
to
add padding on string data. The default value is false
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#bucket_folder ⇒ String
An optional parameter to set a folder name in the S3 bucket. If
provided, tables are created in the path
bucketFolder/schema_name/table_name/
. If this parameter isn't
specified, then the path used is schema_name/table_name/
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#bucket_name ⇒ String
The name of the S3 bucket.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#canned_acl_for_objects ⇒ String
A value that enables DMS to specify a predefined (canned) access control list for objects created in an Amazon S3 bucket as .csv or .parquet files. For more information about Amazon S3 canned ACLs, see Canned ACL in the Amazon S3 Developer Guide.
The default value is NONE. Valid values include NONE, PRIVATE, PUBLIC_READ, PUBLIC_READ_WRITE, AUTHENTICATED_READ, AWS_EXEC_READ, BUCKET_OWNER_READ, and BUCKET_OWNER_FULL_CONTROL.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#cdc_inserts_and_updates ⇒ Boolean
A value that enables a change data capture (CDC) load to write
INSERT and UPDATE operations to .csv or .parquet (columnar storage)
output files. The default setting is false
, but when
CdcInsertsAndUpdates
is set to true
or y
, only INSERTs and
UPDATEs from the source database are migrated to the .csv or
.parquet file.
DMS supports the use of the .parquet files in versions 3.4.7 and later.
How these INSERTs and UPDATEs are recorded depends on the value of
the IncludeOpForFullLoad
parameter. If IncludeOpForFullLoad
is
set to true
, the first field of every CDC record is set to either
I
or U
to indicate INSERT and UPDATE operations at the source.
But if IncludeOpForFullLoad
is set to false
, CDC records are
written without an indication of INSERT or UPDATE operations at the
source. For more information about how these settings work together,
see Indicating Source DB Operations in Migrated S3 Data in the
Database Migration Service User Guide..
CdcInsertsAndUpdates
parameter in
versions 3.3.1 and later.
CdcInsertsOnly
and CdcInsertsAndUpdates
can't both be set to
true
for the same endpoint. Set either CdcInsertsOnly
or
CdcInsertsAndUpdates
to true
for the same endpoint, but not
both.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#cdc_inserts_only ⇒ Boolean
A value that enables a change data capture (CDC) load to write only
INSERT operations to .csv or columnar storage (.parquet) output
files. By default (the false
setting), the first field in a .csv
or .parquet record contains the letter I (INSERT), U (UPDATE), or D
(DELETE). These values indicate whether the row was inserted,
updated, or deleted at the source database for a CDC load to the
target.
If CdcInsertsOnly
is set to true
or y
, only INSERTs from the
source database are migrated to the .csv or .parquet file. For .csv
format only, how these INSERTs are recorded depends on the value of
IncludeOpForFullLoad
. If IncludeOpForFullLoad
is set to true
,
the first field of every CDC record is set to I to indicate the
INSERT operation at the source. If IncludeOpForFullLoad
is set to
false
, every CDC record is written without a first field to
indicate the INSERT operation at the source. For more information
about how these settings work together, see Indicating Source DB
Operations in Migrated S3 Data in the Database Migration
Service User Guide..
CdcInsertsOnly
and IncludeOpForFullLoad
parameters in versions
3.1.4 and later.
CdcInsertsOnly
and CdcInsertsAndUpdates
can't both be set to
true
for the same endpoint. Set either CdcInsertsOnly
or
CdcInsertsAndUpdates
to true
for the same endpoint, but not
both.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#cdc_max_batch_interval ⇒ Integer
Maximum length of the interval, defined in seconds, after which to output a file to Amazon S3.
When CdcMaxBatchInterval
and CdcMinFileSize
are both specified,
the file write is triggered by whichever parameter condition is met
first within an DMS CloudFormation template.
The default value is 60 seconds.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#cdc_min_file_size ⇒ Integer
Minimum file size, defined in kilobytes, to reach for a file output to Amazon S3.
When CdcMinFileSize
and CdcMaxBatchInterval
are both specified,
the file write is triggered by whichever parameter condition is met
first within an DMS CloudFormation template.
The default value is 32 MB.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#cdc_path ⇒ String
Specifies the folder path of CDC files. For an S3 source, this
setting is required if a task captures change data; otherwise, it's
optional. If CdcPath
is set, DMS reads CDC files from this path
and replicates the data changes to the target endpoint. For an S3
target if you set PreserveTransactions
to true
, DMS
verifies that you have set this parameter to a folder path on your
S3 target where DMS can save the transaction order for the CDC load.
DMS creates this CDC folder path in either your S3 target working
directory or the S3 target location specified by BucketFolder
and BucketName
.
For example, if you specify CdcPath
as MyChangedData
, and you
specify BucketName
as MyTargetBucket
but do not specify
BucketFolder
, DMS creates the CDC folder path following:
MyTargetBucket/MyChangedData
.
If you specify the same CdcPath
, and you specify BucketName
as
MyTargetBucket
and BucketFolder
as MyTargetData
, DMS creates
the CDC folder path following:
MyTargetBucket/MyTargetData/MyChangedData
.
For more information on CDC including transaction order on an S3 target, see Capturing data changes (CDC) including transaction order on the S3 target.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#compression_type ⇒ String
An optional parameter to use GZIP to compress the target files. Set to GZIP to compress the target files. Either set this parameter to NONE (the default) or don't use it to leave the files uncompressed. This parameter applies to both .csv and .parquet file formats.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#csv_delimiter ⇒ String
The delimiter used to separate columns in the .csv file for both source and target. The default is a comma.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#csv_no_sup_value ⇒ String
This setting only applies if your Amazon S3 output files during a
change data capture (CDC) load are written in .csv format. If
UseCsvNoSupValue
is set to true, specify a string value that
you want DMS to use for all columns not included in the supplemental
log. If you do not specify a string value, DMS uses the null value
for these columns regardless of the UseCsvNoSupValue
setting.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#csv_null_value ⇒ String
An optional parameter that specifies how DMS treats null values.
While handling the null value, you can use this parameter to pass a
user-defined string as null when writing to the target. For example,
when target columns are nullable, you can use this option to
differentiate between the empty string value and the null value. So,
if you set this parameter value to the empty string ("" or ''),
DMS treats the empty string as the null value instead of NULL
.
The default value is NULL
. Valid values include any valid string.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#csv_row_delimiter ⇒ String
The delimiter used to separate rows in the .csv file for both source
and target. The default is a carriage return (\n
).
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#data_format ⇒ String
The format of the data that you want to use for output. You can choose one of the following:
csv
: This is a row-based file format with comma-separated values (.csv).parquet
: Apache Parquet (.parquet) is a columnar storage file format that features efficient compression and provides faster query response.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#data_page_size ⇒ Integer
The size of one data page in bytes. This parameter defaults to 1024 * 1024 bytes (1 MiB). This number is used for .parquet file format only.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#date_partition_delimiter ⇒ String
Specifies a date separating delimiter to use during folder
partitioning. The default value is SLASH
. Use this parameter when
DatePartitionedEnabled
is set to true
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#date_partition_enabled ⇒ Boolean
When set to true
, this parameter partitions S3 bucket folders
based on transaction commit dates. The default value is false
. For
more information about date-based folder partitioning, see Using
date-based folder partitioning.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#date_partition_sequence ⇒ String
Identifies the sequence of the date format to use during folder
partitioning. The default value is YYYYMMDD
. Use this parameter
when DatePartitionedEnabled
is set to true
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#date_partition_timezone ⇒ String
When creating an S3 target endpoint, set DatePartitionTimezone
to
convert the current UTC time into a specified time zone. The
conversion occurs when a date partition folder is created and a CDC
filename is generated. The time zone format is Area/Location. Use
this parameter when DatePartitionedEnabled
is set to true
, as
shown in the following example.
s3-settings='\{"DatePartitionEnabled": true,
"DatePartitionSequence": "YYYYMMDDHH", "DatePartitionDelimiter":
"SLASH", "DatePartitionTimezone":"Asia/Seoul", "BucketName":
"dms-nattarat-test"\}'
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#dict_page_size_limit ⇒ Integer
The maximum size of an encoded dictionary page of a column. If the
dictionary page exceeds this, this column is stored using an
encoding type of PLAIN
. This parameter defaults to 1024 * 1024
bytes (1 MiB), the maximum size of a dictionary page before it
reverts to PLAIN
encoding. This size is used for .parquet file
format only.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#enable_statistics ⇒ Boolean
A value that enables statistics for Parquet pages and row groups.
Choose true
to enable statistics, false
to disable. Statistics
include NULL
, DISTINCT
, MAX
, and MIN
values. This parameter
defaults to true
. This value is used for .parquet file format
only.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#encoding_type ⇒ String
The type of encoding you are using:
RLE_DICTIONARY
uses a combination of bit-packing and run-length encoding to store repeated values more efficiently. This is the default.PLAIN
doesn't use encoding at all. Values are stored as they are.PLAIN_DICTIONARY
builds a dictionary of the values encountered in a given column. The dictionary is stored in a dictionary page for each column chunk.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#encryption_mode ⇒ String
The type of server-side encryption that you want to use for your
data. This encryption type is part of the endpoint settings or the
extra connections attributes for Amazon S3. You can choose either
SSE_S3
(the default) or SSE_KMS
.
ModifyEndpoint
operation, you can change the existing
value of the EncryptionMode
parameter from SSE_KMS
to SSE_S3
.
But you can’t change the existing value from SSE_S3
to SSE_KMS
.
To use SSE_S3
, you need an Identity and Access Management (IAM)
role with permission to allow "arn:aws:s3:::dms-*"
to use the
following actions:
s3:CreateBucket
s3:ListBucket
s3:DeleteBucket
s3:GetBucketLocation
s3:GetObject
s3:PutObject
s3:DeleteObject
s3:GetObjectVersion
s3:GetBucketPolicy
s3:PutBucketPolicy
s3:DeleteBucketPolicy
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#expected_bucket_owner ⇒ String
To specify a bucket owner and prevent sniping, you can use the
ExpectedBucketOwner
endpoint setting.
Example: --s3-settings='\{"ExpectedBucketOwner":
"AWS_Account_ID"\}'
When you make a request to test a connection or perform a migration, S3 checks the account ID of the bucket owner against the specified parameter.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#external_table_definition ⇒ String
Specifies how tables are defined in the S3 source files only.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#glue_catalog_generation ⇒ Boolean
When true, allows Glue to catalog your S3 bucket. Creating an Glue catalog lets you use Athena to query your data.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#ignore_header_rows ⇒ Integer
When this value is set to 1, DMS ignores the first row header in a .csv file. A value of 1 turns on the feature; a value of 0 turns off the feature.
The default is 0.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#include_op_for_full_load ⇒ Boolean
A value that enables a full load to write INSERT operations to the comma-separated value (.csv) or .parquet output files only to indicate how the rows were added to the source database.
IncludeOpForFullLoad
parameter in versions 3.1.4
and later.
DMS supports the use of the .parquet files with the
IncludeOpForFullLoad
parameter in versions 3.4.7 and later.
For full load, records can only be inserted. By default (the false
setting), no information is recorded in these output files for a
full load to indicate that the rows were inserted at the source
database. If IncludeOpForFullLoad
is set to true
or y
, the
INSERT is recorded as an I annotation in the first field of the .csv
file. This allows the format of your target records from a full load
to be consistent with the target records from a CDC load.
CdcInsertsOnly
and the
CdcInsertsAndUpdates
parameters for output to .csv files only. For
more information about how these settings work together, see
Indicating Source DB Operations in Migrated S3 Data in the
Database Migration Service User Guide..
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#max_file_size ⇒ Integer
A value that specifies the maximum size (in KB) of any .csv file to be created while migrating to an S3 target during full load.
The default value is 1,048,576 KB (1 GB). Valid values include 1 to 1,048,576.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#parquet_timestamp_in_millisecond ⇒ Boolean
A value that specifies the precision of any TIMESTAMP
column
values that are written to an Amazon S3 object file in .parquet
format.
ParquetTimestampInMillisecond
parameter in
versions 3.1.4 and later.
When ParquetTimestampInMillisecond
is set to true
or y
, DMS
writes all TIMESTAMP
columns in a .parquet formatted file with
millisecond precision. Otherwise, DMS writes them with microsecond
precision.
Currently, Amazon Athena and Glue can handle only millisecond
precision for TIMESTAMP
values. Set this parameter to true
for
S3 endpoint object files that are .parquet formatted only if you
plan to query or process the data with Athena or Glue.
TIMESTAMP
column values written to an S3 file in
.csv format with microsecond precision.
Setting ParquetTimestampInMillisecond
has no effect on the string
format of the timestamp column value that is inserted by setting the
TimestampColumnName
parameter.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#parquet_version ⇒ String
The version of the Apache Parquet format that you want to use:
parquet_1_0
(the default) or parquet_2_0
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#preserve_transactions ⇒ Boolean
If set to true
, DMS saves the transaction order for a change data
capture (CDC) load on the Amazon S3 target specified by CdcPath
. For more information, see Capturing data changes (CDC)
including transaction order on the S3 target.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#rfc_4180 ⇒ Boolean
For an S3 source, when this value is set to true
or y
, each
leading double quotation mark has to be followed by an ending double
quotation mark. This formatting complies with RFC 4180. When this
value is set to false
or n
, string literals are copied to the
target as is. In this case, a delimiter (row or column) signals the
end of the field. Thus, you can't use a delimiter as part of the
string, because it signals the end of the value.
For an S3 target, an optional parameter used to set behavior to
comply with RFC 4180 for data migrated to Amazon S3 using .csv file
format only. When this value is set to true
or y
using Amazon S3
as a target, if the data has quotation marks or newline characters
in it, DMS encloses the entire column with an additional pair of
double quotation marks ("). Every quotation mark within the data is
repeated twice.
The default value is true
. Valid values include true
, false
,
y
, and n
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#row_group_length ⇒ Integer
The number of rows in a row group. A smaller row group size provides faster reads. But as the number of row groups grows, the slower writes become. This parameter defaults to 10,000 rows. This number is used for .parquet file format only.
If you choose a value larger than the maximum, RowGroupLength
is
set to the max row group length in bytes (64 * 1024 * 1024).
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#server_side_encryption_kms_key_id ⇒ String
If you are using SSE_KMS
for the EncryptionMode
, provide the KMS
key ID. The key that you use needs an attached policy that enables
Identity and Access Management (IAM) user permissions and allows use
of the key.
Here is a CLI example: aws dms create-endpoint
--endpoint-identifier value --endpoint-type target --engine-name s3
--s3-settings
ServiceAccessRoleArn=value,BucketFolder=value,BucketName=value,EncryptionMode=SSE_KMS,ServerSideEncryptionKmsKeyId=value
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#service_access_role_arn ⇒ String
The Amazon Resource Name (ARN) used by the service to access the IAM
role. The role must allow the iam:PassRole
action. It is a
required parameter that enables DMS to write and read objects from
an S3 bucket.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#timestamp_column_name ⇒ String
A value that when nonblank causes DMS to add a column with timestamp information to the endpoint data for an Amazon S3 target.
TimestampColumnName
parameter in versions 3.1.4
and later.
DMS includes an additional STRING
column in the .csv or .parquet
object files of your migrated data when you set
TimestampColumnName
to a nonblank value.
For a full load, each row of this timestamp column contains a timestamp for when the data was transferred from the source to the target by DMS.
For a change data capture (CDC) load, each row of the timestamp column contains the timestamp for the commit of that row in the source database.
The string format for this timestamp column value is yyyy-MM-dd
HH:mm:ss.SSSSSS
. By default, the precision of this value is in
microseconds. For a CDC load, the rounding of the precision depends
on the commit timestamp supported by DMS for the source database.
When the AddColumnName
parameter is set to true
, DMS also
includes a name for the timestamp column that you set with
TimestampColumnName
.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#use_csv_no_sup_value ⇒ Boolean
This setting applies if the S3 output files during a change data
capture (CDC) load are written in .csv format. If set to true
for
columns not included in the supplemental log, DMS uses the value
specified by CsvNoSupValue
. If not set or set to false
,
DMS uses the null value for these columns.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |
#use_task_start_time_for_full_load_timestamp ⇒ Boolean
When set to true, this parameter uses the task start time as the
timestamp column value instead of the time data is written to
target. For full load, when useTaskStartTimeForFullLoadTimestamp
is set to true
, each row of the timestamp column contains the task
start time. For CDC loads, each row of the timestamp column contains
the transaction commit time.
When useTaskStartTimeForFullLoadTimestamp
is set to false
, the
full load timestamp in the timestamp column increments with the time
data arrives at the target.
12520 12521 12522 12523 12524 12525 12526 12527 12528 12529 12530 12531 12532 12533 12534 12535 12536 12537 12538 12539 12540 12541 12542 12543 12544 12545 12546 12547 12548 12549 12550 12551 12552 12553 12554 12555 12556 12557 12558 12559 12560 12561 12562 12563 12564 |
# File 'gems/aws-sdk-databasemigrationservice/lib/aws-sdk-databasemigrationservice/types.rb', line 12520 class S3Settings < Struct.new( :service_access_role_arn, :external_table_definition, :csv_row_delimiter, :csv_delimiter, :bucket_folder, :bucket_name, :compression_type, :encryption_mode, :server_side_encryption_kms_key_id, :data_format, :encoding_type, :dict_page_size_limit, :row_group_length, :data_page_size, :parquet_version, :enable_statistics, :include_op_for_full_load, :cdc_inserts_only, :timestamp_column_name, :parquet_timestamp_in_millisecond, :cdc_inserts_and_updates, :date_partition_enabled, :date_partition_sequence, :date_partition_delimiter, :use_csv_no_sup_value, :csv_no_sup_value, :preserve_transactions, :cdc_path, :use_task_start_time_for_full_load_timestamp, :canned_acl_for_objects, :add_column_name, :cdc_max_batch_interval, :cdc_min_file_size, :csv_null_value, :ignore_header_rows, :max_file_size, :rfc_4180, :date_partition_timezone, :add_trailing_padding_character, :expected_bucket_owner, :glue_catalog_generation) SENSITIVE = [] include Aws::Structure end |