AWS Tools for Windows PowerShell
Command Reference

AWS services or capabilities described in AWS Documentation may vary by region/location. Click Getting Started with Amazon AWS to see specific differences applicable to the China (Beijing) Region.

Synopsis

Calls the Amazon S3 Tables StartTableMigration API operation.

Syntax

Start-S3TTableMigration
-MigrationOutput_Bucket <String>
-Source_GlueTable_CatalogId <String>
-DestinationTable_Properties_CompressionCodec <CompressionCodec>
-Source_GlueTable_DatabaseName <String>
-DestinationTable_Properties_DictionaryEncoding <Boolean>
-DestinationTable_IcebergSortOrder_Field <IcebergSortField[]>
-DestinationTable_Schema_Field <SchemaV2Field[]>
-DestinationTable_Properties_IcebergFormatVersion <Int32>
-DestinationTable_Schema_IdentifierFieldId <Int32[]>
-DataMigration_MigrationPolicies_IncompatibleValuesPolicy <IncompatibleValuesPolicy>
-DestinationTable_EncryptionConfiguration_KmsKeyArn <String>
-DataMigration_MaintenanceSettings_IcebergSnapshotManagement_MaxSnapshotAgeHour <Int32>
-DataMigration_MaintenanceSettings_IcebergSnapshotManagement_MinSnapshotsToKeep <Int32>
-DestinationTable_Name <String>
-DestinationTable_Namespace <String>
-MigrationOutput_Prefixes_ObjectCompatibilityReport <String>
-MigrationOutput_Prefixes_ObjectMigrationReport <String>
-DestinationTable_IcebergSortOrder_OrderId <Int32>
-ReviewMigrationPlan <Boolean>
-RoleARN <String>
-DestinationTable_Schema_SchemaId <Int32>
-SourceType <MigrationSourceType>
-DestinationTable_EncryptionConfiguration_SseAlgorithm <SSEAlgorithm>
-DestinationTable_StorageClassConfiguration_StorageClass <StorageClass>
-DataMigration_MaintenanceSettings_IcebergCompaction_Strategy <IcebergCompactionStrategy>
-DestinationTable_TableBucketARN <String>
-MigrationOutput_Prefixes_TableCompatibilityReport <String>
-Source_GlueTable_TableName <String>
-DestinationTable_Tag <Hashtable>
-DataMigration_MaintenanceSettings_IcebergCompaction_TargetFileSizeMB <Int32>
-DestinationTable_Schema_Type <SchemaV2FieldType>
-DataMigration_MigrationPolicies_UnmatchedSourceFieldsPolicy <UnmatchedSourceFieldsPolicy>
-DestinationTable_Properties_WriteFormat <DataFileFormat>
-Select <String>
-Force <SwitchParameter>
-ClientConfig <AmazonS3TablesConfig>

Description

Starts migrating an Amazon Web Services Glue Data Catalog table into a new table in a table bucket. The migration runs asynchronously: this operation returns the migration ID and its initial state, and you track progress with GetTableMigration. The namespace must already exist, and a table with the specified name must not. The migration creates the table during planning and commits data to it as it runs. Until you call CompleteTableMigration, you can read the table but you can't write to it. If the migration fails or is cancelled, the table and the data already committed to it remain. The migration first checks the source and the destination, and writes the table compatibility and object compatibility reports to the bucket that you specify in migrationOutput. If you set reviewMigrationPlan to true, the migration waits in the pendingReview state so that you can review the reports, and continues only when you call UpdateTableMigration with requestedState set to queued. If you don't continue or cancel the migration within 30 days, the migration fails. Otherwise it continues to migrate the data, and writes the object migration report when the migration ends. If planning fails, the migration writes no object migration report.
Permissions
  • You must have the s3tables:StartTableMigration permission to use this operation.
  • You must have the iam:PassRole permission on the role that you specify in roleARN, with a condition that allows passing it to migration.s3tables.amazonaws.com. The associated resource is the table bucket.
  • If destinationTable specifies SSE-KMS encryption, you must have the kms:DescribeKey and kms:GenerateDataKey permissions on the KMS key. You must also grant the S3 Tables maintenance principal access to the key. For more information, see Permissions requirements for S3 Tables SSE-KMS encryption.

Parameters

-ClientConfig <AmazonS3TablesConfig>
Amazon.PowerShell.Cmdlets.S3T.AmazonS3TablesClientCmdlet.ClientConfig
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DataMigration_MaintenanceSettings_IcebergCompaction_Strategy <IcebergCompactionStrategy>
The compaction strategy. The only valid value is binpack, which is the default.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DataMigration_MaintenanceSettings_IcebergCompaction_TargetFileSizeMB <Int32>
The target size, from 64 to 512 MB, of the data files that the migration writes, and of the files that compaction writes after you complete the migration. The default is 512.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DataMigration_MaintenanceSettings_IcebergSnapshotManagement_MaxSnapshotAgeHour <Int32>
The maximum age, in hours, of a snapshot before it can be expired. The default is 120.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesDataMigration_MaintenanceSettings_IcebergSnapshotManagement_MaxSnapshotAgeHours
-DataMigration_MaintenanceSettings_IcebergSnapshotManagement_MinSnapshotsToKeep <Int32>
The minimum number of snapshots to keep. The value must be 3 or more. The default is 3.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DataMigration_MigrationPolicies_IncompatibleValuesPolicy <IncompatibleValuesPolicy>
The migration policy for a source value that the destination column's type can't hold. Valid values:
  • fail - Fails the migration. This is the default.
  • setNull - Writes null for a value that the destination column's type can't hold. If a column's type in a source object is incompatible with the destination column's type, the column is null for that whole object.
A required column, or a type that migration doesn't support, fails the migration under either value.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DataMigration_MigrationPolicies_UnmatchedSourceFieldsPolicy <UnmatchedSourceFieldsPolicy>
The migration policy for a source field that has no matching column in the destination schema. Valid values:
  • fail - Fails the migration. This is the default.
  • drop - Leaves source columns that the destination schema doesn't have out of the destination table, and doesn't migrate source objects outside the Amazon Web Services Glue table's partition locations.
A destination schema that leaves out a partition column or a field of a struct column fails the migration under either value.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_EncryptionConfiguration_KmsKeyArn <String>
The Amazon Resource Name (ARN) of the KMS key to use for encryption. This field is required only when sseAlgorithm is set to aws:kms.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_EncryptionConfiguration_SseAlgorithm <SSEAlgorithm>
The server-side encryption algorithm to use. Valid values are AES256 for S3-managed encryption keys, or aws:kms for Amazon Web Services KMS-managed encryption keys. If you choose SSE-KMS encryption you must grant the S3 Tables maintenance principal access to your KMS key. For more information, see Permissions requirements for S3 Tables SSE-KMS encryption.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_IcebergSortOrder_Field <IcebergSortField[]>
The list of sort fields that define how data is sorted within files. Each field specifies a source field, sort direction, and null ordering. This field is required if writeOrder is provided. Starting with version 4 of the SDK this property will default to null. If no data for this property is returned from the service the property will also be null. This was changed to improve performance and allow the SDK and caller to distinguish between a property not set or a property being empty to clear out a value. To retain the previous SDK behavior set the AWSConfigs.InitializeCollections static property to true.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesDestinationTable_IcebergSortOrder_Fields
-DestinationTable_IcebergSortOrder_OrderId <Int32>
The unique identifier for this sort order. If not specified, defaults to 1. The order ID is used by Apache Iceberg to track sort order evolution.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Name <String>
The name of the table to create.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Namespace <String>
The namespace to create the table in.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Properties_CompressionCodec <CompressionCodec>
The compression codec of the data files. The default is zstd.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Properties_DictionaryEncoding <Boolean>
Specifies whether the data files use dictionary encoding. The default is true.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Properties_IcebergFormatVersion <Int32>
The Iceberg format version of the destination table. Valid values are 2 and 3. The default is 2.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Properties_WriteFormat <DataFileFormat>
The file format of the data files. The only supported format is parquet, the default.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Schema_Field <SchemaV2Field[]>
The schema fields for the table. Each field defines a column in the table, including its name, type, and whether it is required. Starting with version 4 of the SDK this property will default to null. If no data for this property is returned from the service the property will also be null. This was changed to improve performance and allow the SDK and caller to distinguish between a property not set or a property being empty to clear out a value. To retain the previous SDK behavior set the AWSConfigs.InitializeCollections static property to true.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesDestinationTable_Schema_Fields
-DestinationTable_Schema_IdentifierFieldId <Int32[]>
A list of field IDs that are used as the identifier fields for the table. Identifier fields uniquely identify a row in the table. Starting with version 4 of the SDK this property will default to null. If no data for this property is returned from the service the property will also be null. This was changed to improve performance and allow the SDK and caller to distinguish between a property not set or a property being empty to clear out a value. To retain the previous SDK behavior set the AWSConfigs.InitializeCollections static property to true.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesDestinationTable_Schema_IdentifierFieldIds
-DestinationTable_Schema_SchemaId <Int32>
An optional unique identifier for the schema. Schema IDs are used by Apache Iceberg to track schema evolution.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Schema_Type <SchemaV2FieldType>
The type of the top-level schema, which is always a struct type as defined in the Apache Iceberg specification. This value must be struct.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_StorageClassConfiguration_StorageClass <StorageClass>
The storage class for the table or table bucket. Valid values include storage classes optimized for different access patterns and cost profiles.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_TableBucketARN <String>
The Amazon Resource Name (ARN) of the table bucket to create the table in.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)
-DestinationTable_Tag <Hashtable>
The tags to add to the table. A tag is a key-value pair that you apply to your resources. For more information, see Tagging for cost allocation or attribute-based access control (ABAC). If you specify tags, the migration role must have the s3tables:TagResource permission. Starting with version 4 of the SDK this property will default to null. If no data for this property is returned from the service the property will also be null. This was changed to improve performance and allow the SDK and caller to distinguish between a property not set or a property being empty to clear out a value. To retain the previous SDK behavior set the AWSConfigs.InitializeCollections static property to true.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesDestinationTable_Tags
-Force <SwitchParameter>
This parameter overrides confirmation prompts to force the cmdlet to continue its operation. This parameter should always be used with caution.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-MigrationOutput_Bucket <String>
The general purpose bucket that the migration writes every report to.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)
-MigrationOutput_Prefixes_ObjectCompatibilityReport <String>
The key prefix for the object compatibility report, up to 512 bytes. If not specified, the report's keys start at the root of the bucket.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-MigrationOutput_Prefixes_ObjectMigrationReport <String>
The key prefix for the object migration report, up to 512 bytes. If not specified, the report's keys start at the root of the bucket.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-MigrationOutput_Prefixes_TableCompatibilityReport <String>
The key prefix for the table compatibility report, up to 512 bytes. If not specified, the report's keys start at the root of the bucket.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-ReviewMigrationPlan <Boolean>
Specifies whether the migration waits in the pendingReview state, so that you can review the compatibility reports before it migrates any data. If true, the migration continues when you call UpdateTableMigration with requestedState set to queued. If you don't continue or cancel the migration within 30 days, the migration fails. The default is false.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-RoleARN <String>
The Amazon Resource Name (ARN) of the IAM role that the migration uses to read the source table and its data and to write the destination table and the reports. The role must be in your account, and its trust policy must allow migration.s3tables.amazonaws.com to assume it. You can limit the trust policy to the destination table bucket with the aws:SourceArn and aws:SourceAccount condition keys.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)
-Select <String>
Use the -Select parameter to control the cmdlet output. The default value is '*'. Specifying -Select '*' will result in the cmdlet returning the whole service response (Amazon.S3Tables.Model.StartTableMigrationResponse). Specifying the name of a property of type Amazon.S3Tables.Model.StartTableMigrationResponse will result in that property being returned. Specifying -Select '^ParameterName' will result in the cmdlet returning the selected cmdlet parameter value.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-Source_GlueTable_CatalogId <String>
The ID of the Data Catalog that contains the table. It must be your account ID.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-Source_GlueTable_DatabaseName <String>
The name of the database that contains the table.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-Source_GlueTable_TableName <String>
The name of the table.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-SourceType <MigrationSourceType>
The type of the source. It must match the member that you set in source.
Required?True
Position?Named
Accept pipeline input?True (ByPropertyName)

Common Credential and Region Parameters

-AccessKey <String>
The AWS access key for the user account. This can be a temporary access key if the corresponding session token is supplied to the -SessionToken parameter.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesAK
-Credential <AWSCredentials>
An AWSCredentials object instance containing access and secret key information, and optionally a token for session-based credentials.
Required?False
Position?Named
Accept pipeline input?True (ByValue, ByPropertyName)
-EndpointUrl <String>
The endpoint to make the call against.Note: This parameter is primarily for internal AWS use and is not required/should not be specified for normal usage. The cmdlets normally determine which endpoint to call based on the region specified to the -Region parameter or set as default in the shell (via Set-DefaultAWSRegion). Only specify this parameter if you must direct the call to a specific custom endpoint.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
-NetworkCredential <PSCredential>
Used with SAML-based authentication when ProfileName references a SAML role profile. Contains the network credentials to be supplied during authentication with the configured identity provider's endpoint. This parameter is not required if the user's default network identity can or should be used during authentication.
Required?False
Position?Named
Accept pipeline input?True (ByValue, ByPropertyName)
-ProfileLocation <String>
Used to specify the name and location of the ini-format credential file (shared with the AWS CLI and other AWS SDKs)If this optional parameter is omitted this cmdlet will search the encrypted credential file used by the AWS SDK for .NET and AWS Toolkit for Visual Studio first. If the profile is not found then the cmdlet will search in the ini-format credential file at the default location: (user's home directory)\.aws\credentials.If this parameter is specified then this cmdlet will only search the ini-format credential file at the location given.As the current folder can vary in a shell or during script execution it is advised that you use specify a fully qualified path instead of a relative path.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesAWSProfilesLocation, ProfilesLocation
-ProfileName <String>
The user-defined name of an AWS credentials or SAML-based role profile containing credential information. The profile is expected to be found in the secure credential file shared with the AWS SDK for .NET and AWS Toolkit for Visual Studio. You can also specify the name of a profile stored in the .ini-format credential file used with the AWS CLI and other AWS SDKs.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesStoredCredentials, AWSProfileName
-Region <Object>
The system name of an AWS region or an AWSRegion instance. This governs the endpoint that will be used when calling service operations. Note that the AWS resources referenced in a call are usually region-specific.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesRegionToCall
-SecretKey <String>
The AWS secret key for the user account. This can be a temporary secret key if the corresponding session token is supplied to the -SessionToken parameter.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesSK, SecretAccessKey
-SessionToken <String>
The session token if the access and secret keys are temporary session-based credentials.
Required?False
Position?Named
Accept pipeline input?True (ByPropertyName)
AliasesST

Outputs

Amazon.S3Tables.Model.StartTableMigrationResponse
This cmdlet returns an Amazon.S3Tables.Model.StartTableMigrationResponse object containing multiple properties.
AWS Tools for PowerShell User Guide

Supported Version

AWS Tools for PowerShell: 2.x.y.z