BatchCreatePartition

$result = $client->batchCreatePartition([/* ... */]);
$promise = $client->batchCreatePartitionAsync([/* ... */]);

Creates one or more partitions in a batch operation.

Parameter Syntax

$result = $client->batchCreatePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionInputList' => [ // REQUIRED
        [
            'LastAccessTime' => <integer || string || DateTime>,
            'LastAnalyzedTime' => <integer || string || DateTime>,
            'Parameters' => ['<string>', ...],
            'StorageDescriptor' => [
                'AdditionalLocations' => ['<string>', ...],
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>', // REQUIRED
                        'Parameters' => ['<string>', ...],
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SchemaReference' => [
                    'SchemaId' => [
                        'RegistryName' => '<string>',
                        'SchemaArn' => '<string>',
                        'SchemaName' => '<string>',
                    ],
                    'SchemaVersionId' => '<string>',
                    'SchemaVersionNumber' => <integer>,
                ],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>', // REQUIRED
                        'SortOrder' => <integer>, // REQUIRED
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'PartitionValues' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InternalServiceException:

An internal service error occurred.

EntityNotFoundException:

A specified entity does not exist

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

BatchDeleteConnection

$result = $client->batchDeleteConnection([/* ... */]);
$promise = $client->batchDeleteConnectionAsync([/* ... */]);

Deletes a list of connection definitions from the Data Catalog.

Parameter Syntax

$result = $client->batchDeleteConnection([
    'CatalogId' => '<string>',
    'ConnectionNameList' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        '<NameString>' => [
            'ErrorCode' => '<string>',
            'ErrorMessage' => '<string>',
        ],
        // ...
    ],
    'Succeeded' => ['<string>', ...],
]

Result Details

Errors

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

BatchDeletePartition

$result = $client->batchDeletePartition([/* ... */]);
$promise = $client->batchDeletePartitionAsync([/* ... */]);

Deletes one or more partitions in a batch operation.

Parameter Syntax

$result = $client->batchDeletePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionsToDelete' => [ // REQUIRED
        [
            'Values' => ['<string>', ...], // REQUIRED
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'PartitionValues' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

EntityNotFoundException:

A specified entity does not exist

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

BatchDeleteTable

$result = $client->batchDeleteTable([/* ... */]);
$promise = $client->batchDeleteTableAsync([/* ... */]);

Deletes multiple tables at once.

Parameter Syntax

$result = $client->batchDeleteTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TablesToDelete' => ['<string>', ...], // REQUIRED
    'TransactionId' => '<string>',
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'TableName' => '<string>',
        ],
        // ...
    ],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

EntityNotFoundException:

A specified entity does not exist

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

ResourceNotReadyException:

A resource was not ready for a transaction.

BatchDeleteTableVersion

$result = $client->batchDeleteTableVersion([/* ... */]);
$promise = $client->batchDeleteTableVersionAsync([/* ... */]);

Deletes a specified batch of versions of a table.

Parameter Syntax

$result = $client->batchDeleteTableVersion([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TableName' => '<string>', // REQUIRED
    'VersionIds' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'TableName' => '<string>',
            'VersionId' => '<string>',
        ],
        // ...
    ],
]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

BatchGetBlueprints

$result = $client->batchGetBlueprints([/* ... */]);
$promise = $client->batchGetBlueprintsAsync([/* ... */]);

Retrieves information about a list of blueprints.

Parameter Syntax

$result = $client->batchGetBlueprints([
    'IncludeBlueprint' => true || false,
    'IncludeParameterSpec' => true || false,
    'Names' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Blueprints' => [
        [
            'BlueprintLocation' => '<string>',
            'BlueprintServiceLocation' => '<string>',
            'CreatedOn' => <DateTime>,
            'Description' => '<string>',
            'ErrorMessage' => '<string>',
            'LastActiveDefinition' => [
                'BlueprintLocation' => '<string>',
                'BlueprintServiceLocation' => '<string>',
                'Description' => '<string>',
                'LastModifiedOn' => <DateTime>,
                'ParameterSpec' => '<string>',
            ],
            'LastModifiedOn' => <DateTime>,
            'Name' => '<string>',
            'ParameterSpec' => '<string>',
            'Status' => 'CREATING|ACTIVE|UPDATING|FAILED',
        ],
        // ...
    ],
    'MissingBlueprints' => ['<string>', ...],
]

Result Details

Errors

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

BatchGetCrawlers

$result = $client->batchGetCrawlers([/* ... */]);
$promise = $client->batchGetCrawlersAsync([/* ... */]);

Returns a list of resource metadata for a given list of crawler names. After calling the ListCrawlers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetCrawlers([
    'CrawlerNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Crawlers' => [
        [
            'Classifiers' => ['<string>', ...],
            'Configuration' => '<string>',
            'CrawlElapsedTime' => <integer>,
            'CrawlerSecurityConfiguration' => '<string>',
            'CreationTime' => <DateTime>,
            'DatabaseName' => '<string>',
            'Description' => '<string>',
            'LakeFormationConfiguration' => [
                'AccountId' => '<string>',
                'UseLakeFormationCredentials' => true || false,
            ],
            'LastCrawl' => [
                'ErrorMessage' => '<string>',
                'LogGroup' => '<string>',
                'LogStream' => '<string>',
                'MessagePrefix' => '<string>',
                'StartTime' => <DateTime>,
                'Status' => 'SUCCEEDED|CANCELLED|FAILED',
            ],
            'LastUpdated' => <DateTime>,
            'LineageConfiguration' => [
                'CrawlerLineageSettings' => 'ENABLE|DISABLE',
            ],
            'Name' => '<string>',
            'RecrawlPolicy' => [
                'RecrawlBehavior' => 'CRAWL_EVERYTHING|CRAWL_NEW_FOLDERS_ONLY|CRAWL_EVENT_MODE',
            ],
            'Role' => '<string>',
            'Schedule' => [
                'ScheduleExpression' => '<string>',
                'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
            ],
            'SchemaChangePolicy' => [
                'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
                'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
            ],
            'State' => 'READY|RUNNING|STOPPING',
            'TablePrefix' => '<string>',
            'Targets' => [
                'CatalogTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'DatabaseName' => '<string>',
                        'DlqEventQueueArn' => '<string>',
                        'EventQueueArn' => '<string>',
                        'Tables' => ['<string>', ...],
                    ],
                    // ...
                ],
                'DeltaTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'CreateNativeDeltaTable' => true || false,
                        'DeltaTables' => ['<string>', ...],
                        'WriteManifest' => true || false,
                    ],
                    // ...
                ],
                'DynamoDBTargets' => [
                    [
                        'Path' => '<string>',
                        'scanAll' => true || false,
                        'scanRate' => <float>,
                    ],
                    // ...
                ],
                'HudiTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'Exclusions' => ['<string>', ...],
                        'MaximumTraversalDepth' => <integer>,
                        'Paths' => ['<string>', ...],
                    ],
                    // ...
                ],
                'IcebergTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'Exclusions' => ['<string>', ...],
                        'MaximumTraversalDepth' => <integer>,
                        'Paths' => ['<string>', ...],
                    ],
                    // ...
                ],
                'JdbcTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'EnableAdditionalMetadata' => ['<string>', ...],
                        'Exclusions' => ['<string>', ...],
                        'Path' => '<string>',
                    ],
                    // ...
                ],
                'MongoDBTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'Path' => '<string>',
                        'ScanAll' => true || false,
                    ],
                    // ...
                ],
                'S3Targets' => [
                    [
                        'ConnectionName' => '<string>',
                        'DlqEventQueueArn' => '<string>',
                        'EventQueueArn' => '<string>',
                        'Exclusions' => ['<string>', ...],
                        'Path' => '<string>',
                        'SampleSize' => <integer>,
                    ],
                    // ...
                ],
            ],
            'Version' => <integer>,
        ],
        // ...
    ],
    'CrawlersNotFound' => ['<string>', ...],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

BatchGetCustomEntityTypes

$result = $client->batchGetCustomEntityTypes([/* ... */]);
$promise = $client->batchGetCustomEntityTypesAsync([/* ... */]);

Retrieves the details for the custom patterns specified by a list of names.

Parameter Syntax

$result = $client->batchGetCustomEntityTypes([
    'Names' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'CustomEntityTypes' => [
        [
            'ContextWords' => ['<string>', ...],
            'Name' => '<string>',
            'RegexString' => '<string>',
        ],
        // ...
    ],
    'CustomEntityTypesNotFound' => ['<string>', ...],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

BatchGetDataQualityResult

$result = $client->batchGetDataQualityResult([/* ... */]);
$promise = $client->batchGetDataQualityResultAsync([/* ... */]);

Retrieves a list of data quality results for the specified result IDs.

Parameter Syntax

$result = $client->batchGetDataQualityResult([
    'ResultIds' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Results' => [
        [
            'AnalyzerResults' => [
                [
                    'Description' => '<string>',
                    'EvaluatedMetrics' => [<float>, ...],
                    'EvaluationMessage' => '<string>',
                    'Name' => '<string>',
                ],
                // ...
            ],
            'CompletedOn' => <DateTime>,
            'DataSource' => [
                'GlueTable' => [
                    'AdditionalOptions' => ['<string>', ...],
                    'CatalogId' => '<string>',
                    'ConnectionName' => '<string>',
                    'DatabaseName' => '<string>',
                    'TableName' => '<string>',
                ],
            ],
            'EvaluationContext' => '<string>',
            'JobName' => '<string>',
            'JobRunId' => '<string>',
            'Observations' => [
                [
                    'Description' => '<string>',
                    'MetricBasedObservation' => [
                        'MetricName' => '<string>',
                        'MetricValues' => [
                            'ActualValue' => <float>,
                            'ExpectedValue' => <float>,
                            'LowerLimit' => <float>,
                            'UpperLimit' => <float>,
                        ],
                        'NewRules' => ['<string>', ...],
                        'StatisticId' => '<string>',
                    ],
                ],
                // ...
            ],
            'ProfileId' => '<string>',
            'ResultId' => '<string>',
            'RuleResults' => [
                [
                    'Description' => '<string>',
                    'EvaluatedMetrics' => [<float>, ...],
                    'EvaluatedRule' => '<string>',
                    'EvaluationMessage' => '<string>',
                    'Name' => '<string>',
                    'Result' => 'PASS|FAIL|ERROR',
                ],
                // ...
            ],
            'RulesetEvaluationRunId' => '<string>',
            'RulesetName' => '<string>',
            'Score' => <float>,
            'StartedOn' => <DateTime>,
        ],
        // ...
    ],
    'ResultsNotFound' => ['<string>', ...],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

BatchGetDevEndpoints

$result = $client->batchGetDevEndpoints([/* ... */]);
$promise = $client->batchGetDevEndpointsAsync([/* ... */]);

Returns a list of resource metadata for a given list of development endpoint names. After calling the ListDevEndpoints operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetDevEndpoints([
    'DevEndpointNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'DevEndpoints' => [
        [
            'Arguments' => ['<string>', ...],
            'AvailabilityZone' => '<string>',
            'CreatedTimestamp' => <DateTime>,
            'EndpointName' => '<string>',
            'ExtraJarsS3Path' => '<string>',
            'ExtraPythonLibsS3Path' => '<string>',
            'FailureReason' => '<string>',
            'GlueVersion' => '<string>',
            'LastModifiedTimestamp' => <DateTime>,
            'LastUpdateStatus' => '<string>',
            'NumberOfNodes' => <integer>,
            'NumberOfWorkers' => <integer>,
            'PrivateAddress' => '<string>',
            'PublicAddress' => '<string>',
            'PublicKey' => '<string>',
            'PublicKeys' => ['<string>', ...],
            'RoleArn' => '<string>',
            'SecurityConfiguration' => '<string>',
            'SecurityGroupIds' => ['<string>', ...],
            'Status' => '<string>',
            'SubnetId' => '<string>',
            'VpcId' => '<string>',
            'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
            'YarnEndpointAddress' => '<string>',
            'ZeppelinRemoteSparkInterpreterPort' => <integer>,
        ],
        // ...
    ],
    'DevEndpointsNotFound' => ['<string>', ...],
]

Result Details

Errors

AccessDeniedException:

Access to a resource was denied.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

BatchGetJobs

$result = $client->batchGetJobs([/* ... */]);
$promise = $client->batchGetJobsAsync([/* ... */]);

Returns a list of resource metadata for a given list of job names. After calling the ListJobs operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetJobs([
    'JobNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Jobs' => [
        [
            'AllocatedCapacity' => <integer>,
            'CodeGenConfigurationNodes' => [
                '<NodeId>' => [
                    'Aggregate' => [
                        'Aggs' => [
                            [
                                'AggFunc' => 'avg|countDistinct|count|first|last|kurtosis|max|min|skewness|stddev_samp|stddev_pop|sum|sumDistinct|var_samp|var_pop',
                                'Column' => ['<string>', ...],
                            ],
                            // ...
                        ],
                        'Groups' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'AmazonRedshiftSource' => [
                        'Data' => [
                            'AccessType' => '<string>',
                            'Action' => '<string>',
                            'AdvancedOptions' => [
                                [
                                    'Key' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'CatalogDatabase' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'CatalogRedshiftSchema' => '<string>',
                            'CatalogRedshiftTable' => '<string>',
                            'CatalogTable' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'Connection' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'CrawlerConnection' => '<string>',
                            'IamRole' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'MergeAction' => '<string>',
                            'MergeClause' => '<string>',
                            'MergeWhenMatched' => '<string>',
                            'MergeWhenNotMatched' => '<string>',
                            'PostAction' => '<string>',
                            'PreAction' => '<string>',
                            'SampleQuery' => '<string>',
                            'Schema' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'SelectedColumns' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'SourceType' => '<string>',
                            'StagingTable' => '<string>',
                            'Table' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'TablePrefix' => '<string>',
                            'TableSchema' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'TempDir' => '<string>',
                            'Upsert' => true || false,
                        ],
                        'Name' => '<string>',
                    ],
                    'AmazonRedshiftTarget' => [
                        'Data' => [
                            'AccessType' => '<string>',
                            'Action' => '<string>',
                            'AdvancedOptions' => [
                                [
                                    'Key' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'CatalogDatabase' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'CatalogRedshiftSchema' => '<string>',
                            'CatalogRedshiftTable' => '<string>',
                            'CatalogTable' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'Connection' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'CrawlerConnection' => '<string>',
                            'IamRole' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'MergeAction' => '<string>',
                            'MergeClause' => '<string>',
                            'MergeWhenMatched' => '<string>',
                            'MergeWhenNotMatched' => '<string>',
                            'PostAction' => '<string>',
                            'PreAction' => '<string>',
                            'SampleQuery' => '<string>',
                            'Schema' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'SelectedColumns' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'SourceType' => '<string>',
                            'StagingTable' => '<string>',
                            'Table' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'TablePrefix' => '<string>',
                            'TableSchema' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'TempDir' => '<string>',
                            'Upsert' => true || false,
                        ],
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'ApplyMapping' => [
                        'Inputs' => ['<string>', ...],
                        'Mapping' => [
                            [
                                'Children' => [...], // RECURSIVE
                                'Dropped' => true || false,
                                'FromPath' => ['<string>', ...],
                                'FromType' => '<string>',
                                'ToKey' => '<string>',
                                'ToType' => '<string>',
                            ],
                            // ...
                        ],
                        'Name' => '<string>',
                    ],
                    'AthenaConnectorSource' => [
                        'ConnectionName' => '<string>',
                        'ConnectionTable' => '<string>',
                        'ConnectionType' => '<string>',
                        'ConnectorName' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'SchemaName' => '<string>',
                    ],
                    'CatalogDeltaSource' => [
                        'AdditionalDeltaOptions' => ['<string>', ...],
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Table' => '<string>',
                    ],
                    'CatalogHudiSource' => [
                        'AdditionalHudiOptions' => ['<string>', ...],
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Table' => '<string>',
                    ],
                    'CatalogKafkaSource' => [
                        'DataPreviewOptions' => [
                            'PollingTime' => <integer>,
                            'RecordPollingLimit' => <integer>,
                        ],
                        'Database' => '<string>',
                        'DetectSchema' => true || false,
                        'Name' => '<string>',
                        'StreamingOptions' => [
                            'AddRecordTimestamp' => '<string>',
                            'Assign' => '<string>',
                            'BootstrapServers' => '<string>',
                            'Classification' => '<string>',
                            'ConnectionName' => '<string>',
                            'Delimiter' => '<string>',
                            'EmitConsumerLagMetrics' => '<string>',
                            'EndingOffsets' => '<string>',
                            'IncludeHeaders' => true || false,
                            'MaxOffsetsPerTrigger' => <integer>,
                            'MinPartitions' => <integer>,
                            'NumRetries' => <integer>,
                            'PollTimeoutMs' => <integer>,
                            'RetryIntervalMs' => <integer>,
                            'SecurityProtocol' => '<string>',
                            'StartingOffsets' => '<string>',
                            'StartingTimestamp' => <DateTime>,
                            'SubscribePattern' => '<string>',
                            'TopicName' => '<string>',
                        ],
                        'Table' => '<string>',
                        'WindowSize' => <integer>,
                    ],
                    'CatalogKinesisSource' => [
                        'DataPreviewOptions' => [
                            'PollingTime' => <integer>,
                            'RecordPollingLimit' => <integer>,
                        ],
                        'Database' => '<string>',
                        'DetectSchema' => true || false,
                        'Name' => '<string>',
                        'StreamingOptions' => [
                            'AddIdleTimeBetweenReads' => true || false,
                            'AddRecordTimestamp' => '<string>',
                            'AvoidEmptyBatches' => true || false,
                            'Classification' => '<string>',
                            'Delimiter' => '<string>',
                            'DescribeShardInterval' => <integer>,
                            'EmitConsumerLagMetrics' => '<string>',
                            'EndpointUrl' => '<string>',
                            'IdleTimeBetweenReadsInMs' => <integer>,
                            'MaxFetchRecordsPerShard' => <integer>,
                            'MaxFetchTimeInMs' => <integer>,
                            'MaxRecordPerRead' => <integer>,
                            'MaxRetryIntervalMs' => <integer>,
                            'NumRetries' => <integer>,
                            'RetryIntervalMs' => <integer>,
                            'RoleArn' => '<string>',
                            'RoleSessionName' => '<string>',
                            'StartingPosition' => 'latest|trim_horizon|earliest|timestamp',
                            'StartingTimestamp' => <DateTime>,
                            'StreamArn' => '<string>',
                            'StreamName' => '<string>',
                        ],
                        'Table' => '<string>',
                        'WindowSize' => <integer>,
                    ],
                    'CatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'CatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Table' => '<string>',
                    ],
                    'ConnectorDataSource' => [
                        'ConnectionType' => '<string>',
                        'Data' => ['<string>', ...],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'ConnectorDataTarget' => [
                        'ConnectionType' => '<string>',
                        'Data' => ['<string>', ...],
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'CustomCode' => [
                        'ClassName' => '<string>',
                        'Code' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'DirectJDBCSource' => [
                        'ConnectionName' => '<string>',
                        'ConnectionType' => 'sqlserver|mysql|oracle|postgresql|redshift',
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'RedshiftTmpDir' => '<string>',
                        'Table' => '<string>',
                    ],
                    'DirectKafkaSource' => [
                        'DataPreviewOptions' => [
                            'PollingTime' => <integer>,
                            'RecordPollingLimit' => <integer>,
                        ],
                        'DetectSchema' => true || false,
                        'Name' => '<string>',
                        'StreamingOptions' => [
                            'AddRecordTimestamp' => '<string>',
                            'Assign' => '<string>',
                            'BootstrapServers' => '<string>',
                            'Classification' => '<string>',
                            'ConnectionName' => '<string>',
                            'Delimiter' => '<string>',
                            'EmitConsumerLagMetrics' => '<string>',
                            'EndingOffsets' => '<string>',
                            'IncludeHeaders' => true || false,
                            'MaxOffsetsPerTrigger' => <integer>,
                            'MinPartitions' => <integer>,
                            'NumRetries' => <integer>,
                            'PollTimeoutMs' => <integer>,
                            'RetryIntervalMs' => <integer>,
                            'SecurityProtocol' => '<string>',
                            'StartingOffsets' => '<string>',
                            'StartingTimestamp' => <DateTime>,
                            'SubscribePattern' => '<string>',
                            'TopicName' => '<string>',
                        ],
                        'WindowSize' => <integer>,
                    ],
                    'DirectKinesisSource' => [
                        'DataPreviewOptions' => [
                            'PollingTime' => <integer>,
                            'RecordPollingLimit' => <integer>,
                        ],
                        'DetectSchema' => true || false,
                        'Name' => '<string>',
                        'StreamingOptions' => [
                            'AddIdleTimeBetweenReads' => true || false,
                            'AddRecordTimestamp' => '<string>',
                            'AvoidEmptyBatches' => true || false,
                            'Classification' => '<string>',
                            'Delimiter' => '<string>',
                            'DescribeShardInterval' => <integer>,
                            'EmitConsumerLagMetrics' => '<string>',
                            'EndpointUrl' => '<string>',
                            'IdleTimeBetweenReadsInMs' => <integer>,
                            'MaxFetchRecordsPerShard' => <integer>,
                            'MaxFetchTimeInMs' => <integer>,
                            'MaxRecordPerRead' => <integer>,
                            'MaxRetryIntervalMs' => <integer>,
                            'NumRetries' => <integer>,
                            'RetryIntervalMs' => <integer>,
                            'RoleArn' => '<string>',
                            'RoleSessionName' => '<string>',
                            'StartingPosition' => 'latest|trim_horizon|earliest|timestamp',
                            'StartingTimestamp' => <DateTime>,
                            'StreamArn' => '<string>',
                            'StreamName' => '<string>',
                        ],
                        'WindowSize' => <integer>,
                    ],
                    'DropDuplicates' => [
                        'Columns' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'DropFields' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Paths' => [
                            ['<string>', ...],
                            // ...
                        ],
                    ],
                    'DropNullFields' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'NullCheckBoxList' => [
                            'IsEmpty' => true || false,
                            'IsNegOne' => true || false,
                            'IsNullString' => true || false,
                        ],
                        'NullTextList' => [
                            [
                                'Datatype' => [
                                    'Id' => '<string>',
                                    'Label' => '<string>',
                                ],
                                'Value' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    'DynamicTransform' => [
                        'FunctionName' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Parameters' => [
                            [
                                'IsOptional' => true || false,
                                'ListType' => 'str|int|float|complex|bool|list|null',
                                'Name' => '<string>',
                                'Type' => 'str|int|float|complex|bool|list|null',
                                'ValidationMessage' => '<string>',
                                'ValidationRule' => '<string>',
                                'Value' => ['<string>', ...],
                            ],
                            // ...
                        ],
                        'Path' => '<string>',
                        'TransformName' => '<string>',
                        'Version' => '<string>',
                    ],
                    'DynamoDBCatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'EvaluateDataQuality' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Output' => 'PrimaryInput|EvaluationResults',
                        'PublishingOptions' => [
                            'CloudWatchMetricsEnabled' => true || false,
                            'EvaluationContext' => '<string>',
                            'ResultsPublishingEnabled' => true || false,
                            'ResultsS3Prefix' => '<string>',
                        ],
                        'Ruleset' => '<string>',
                        'StopJobOnFailureOptions' => [
                            'StopJobOnFailureTiming' => 'Immediate|AfterDataLoad',
                        ],
                    ],
                    'EvaluateDataQualityMultiFrame' => [
                        'AdditionalDataSources' => ['<string>', ...],
                        'AdditionalOptions' => ['<string>', ...],
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PublishingOptions' => [
                            'CloudWatchMetricsEnabled' => true || false,
                            'EvaluationContext' => '<string>',
                            'ResultsPublishingEnabled' => true || false,
                            'ResultsS3Prefix' => '<string>',
                        ],
                        'Ruleset' => '<string>',
                        'StopJobOnFailureOptions' => [
                            'StopJobOnFailureTiming' => 'Immediate|AfterDataLoad',
                        ],
                    ],
                    'FillMissingValues' => [
                        'FilledPath' => '<string>',
                        'ImputedPath' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'Filter' => [
                        'Filters' => [
                            [
                                'Negated' => true || false,
                                'Operation' => 'EQ|LT|GT|LTE|GTE|REGEX|ISNULL',
                                'Values' => [
                                    [
                                        'Type' => 'COLUMNEXTRACTED|CONSTANT',
                                        'Value' => ['<string>', ...],
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Inputs' => ['<string>', ...],
                        'LogicalOperator' => 'AND|OR',
                        'Name' => '<string>',
                    ],
                    'GovernedCatalogSource' => [
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                        ],
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'PartitionPredicate' => '<string>',
                        'Table' => '<string>',
                    ],
                    'GovernedCatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'SchemaChangePolicy' => [
                            'EnableUpdateCatalog' => true || false,
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                        'Table' => '<string>',
                    ],
                    'JDBCConnectorSource' => [
                        'AdditionalOptions' => [
                            'DataTypeMapping' => ['<string>', ...],
                            'FilterPredicate' => '<string>',
                            'JobBookmarkKeys' => ['<string>', ...],
                            'JobBookmarkKeysSortOrder' => '<string>',
                            'LowerBound' => <integer>,
                            'NumPartitions' => <integer>,
                            'PartitionColumn' => '<string>',
                            'UpperBound' => <integer>,
                        ],
                        'ConnectionName' => '<string>',
                        'ConnectionTable' => '<string>',
                        'ConnectionType' => '<string>',
                        'ConnectorName' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Query' => '<string>',
                    ],
                    'JDBCConnectorTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'ConnectionName' => '<string>',
                        'ConnectionTable' => '<string>',
                        'ConnectionType' => '<string>',
                        'ConnectorName' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'Join' => [
                        'Columns' => [
                            [
                                'From' => '<string>',
                                'Keys' => [
                                    ['<string>', ...],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Inputs' => ['<string>', ...],
                        'JoinType' => 'equijoin|left|right|outer|leftsemi|leftanti',
                        'Name' => '<string>',
                    ],
                    'Merge' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PrimaryKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Source' => '<string>',
                    ],
                    'MicrosoftSQLServerCatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'MicrosoftSQLServerCatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'MySQLCatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'MySQLCatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'OracleSQLCatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'OracleSQLCatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'PIIDetection' => [
                        'EntityTypesToDetect' => ['<string>', ...],
                        'Inputs' => ['<string>', ...],
                        'MaskValue' => '<string>',
                        'Name' => '<string>',
                        'OutputColumnName' => '<string>',
                        'PiiType' => 'RowAudit|RowMasking|ColumnAudit|ColumnMasking',
                        'SampleFraction' => <float>,
                        'ThresholdFraction' => <float>,
                    ],
                    'PostgreSQLCatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'PostgreSQLCatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'Recipe' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'RecipeReference' => [
                            'RecipeArn' => '<string>',
                            'RecipeVersion' => '<string>',
                        ],
                        'RecipeSteps' => [
                            [
                                'Action' => [
                                    'Operation' => '<string>',
                                    'Parameters' => ['<string>', ...],
                                ],
                                'ConditionExpressions' => [
                                    [
                                        'Condition' => '<string>',
                                        'TargetColumn' => '<string>',
                                        'Value' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'RedshiftSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'RedshiftTmpDir' => '<string>',
                        'Table' => '<string>',
                        'TmpDirIAMRole' => '<string>',
                    ],
                    'RedshiftTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'RedshiftTmpDir' => '<string>',
                        'Table' => '<string>',
                        'TmpDirIAMRole' => '<string>',
                        'UpsertRedshiftOptions' => [
                            'ConnectionName' => '<string>',
                            'TableLocation' => '<string>',
                            'UpsertKeys' => ['<string>', ...],
                        ],
                    ],
                    'RelationalCatalogSource' => [
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'Table' => '<string>',
                    ],
                    'RenameField' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'SourcePath' => ['<string>', ...],
                        'TargetPath' => ['<string>', ...],
                    ],
                    'S3CatalogDeltaSource' => [
                        'AdditionalDeltaOptions' => ['<string>', ...],
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Table' => '<string>',
                    ],
                    'S3CatalogHudiSource' => [
                        'AdditionalHudiOptions' => ['<string>', ...],
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Table' => '<string>',
                    ],
                    'S3CatalogSource' => [
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                        ],
                        'Database' => '<string>',
                        'Name' => '<string>',
                        'PartitionPredicate' => '<string>',
                        'Table' => '<string>',
                    ],
                    'S3CatalogTarget' => [
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'SchemaChangePolicy' => [
                            'EnableUpdateCatalog' => true || false,
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                        'Table' => '<string>',
                    ],
                    'S3CsvSource' => [
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                            'EnableSamplePath' => true || false,
                            'SamplePath' => '<string>',
                        ],
                        'CompressionType' => 'gzip|bzip2',
                        'Escaper' => '<string>',
                        'Exclusions' => ['<string>', ...],
                        'GroupFiles' => '<string>',
                        'GroupSize' => '<string>',
                        'MaxBand' => <integer>,
                        'MaxFilesInBand' => <integer>,
                        'Multiline' => true || false,
                        'Name' => '<string>',
                        'OptimizePerformance' => true || false,
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Paths' => ['<string>', ...],
                        'QuoteChar' => 'quote|quillemet|single_quote|disabled',
                        'Recurse' => true || false,
                        'Separator' => 'comma|ctrla|pipe|semicolon|tab',
                        'SkipFirst' => true || false,
                        'WithHeader' => true || false,
                        'WriteHeader' => true || false,
                    ],
                    'S3DeltaCatalogTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'SchemaChangePolicy' => [
                            'EnableUpdateCatalog' => true || false,
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                        'Table' => '<string>',
                    ],
                    'S3DeltaDirectTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'Compression' => 'uncompressed|snappy',
                        'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'NumberTargetPartitions' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Path' => '<string>',
                        'SchemaChangePolicy' => [
                            'Database' => '<string>',
                            'EnableUpdateCatalog' => true || false,
                            'Table' => '<string>',
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                    ],
                    'S3DeltaSource' => [
                        'AdditionalDeltaOptions' => ['<string>', ...],
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                            'EnableSamplePath' => true || false,
                            'SamplePath' => '<string>',
                        ],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Paths' => ['<string>', ...],
                    ],
                    'S3DirectTarget' => [
                        'Compression' => '<string>',
                        'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'NumberTargetPartitions' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Path' => '<string>',
                        'SchemaChangePolicy' => [
                            'Database' => '<string>',
                            'EnableUpdateCatalog' => true || false,
                            'Table' => '<string>',
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                    ],
                    'S3ExcelSource' => [
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                            'EnableSamplePath' => true || false,
                            'SamplePath' => '<string>',
                        ],
                        'CompressionType' => 'snappy|lzo|gzip|brotli|lz4|uncompressed|none',
                        'Exclusions' => ['<string>', ...],
                        'GroupFiles' => '<string>',
                        'GroupSize' => '<string>',
                        'MaxBand' => <integer>,
                        'MaxFilesInBand' => <integer>,
                        'Name' => '<string>',
                        'NumberRows' => <integer>,
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Paths' => ['<string>', ...],
                        'Recurse' => true || false,
                        'SkipFooter' => <integer>,
                    ],
                    'S3GlueParquetTarget' => [
                        'Compression' => 'snappy|lzo|gzip|brotli|lz4|uncompressed|none',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'NumberTargetPartitions' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Path' => '<string>',
                        'SchemaChangePolicy' => [
                            'Database' => '<string>',
                            'EnableUpdateCatalog' => true || false,
                            'Table' => '<string>',
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                    ],
                    'S3HudiCatalogTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'Database' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'SchemaChangePolicy' => [
                            'EnableUpdateCatalog' => true || false,
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                        'Table' => '<string>',
                    ],
                    'S3HudiDirectTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'Compression' => 'gzip|lzo|uncompressed|snappy',
                        'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'NumberTargetPartitions' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Path' => '<string>',
                        'SchemaChangePolicy' => [
                            'Database' => '<string>',
                            'EnableUpdateCatalog' => true || false,
                            'Table' => '<string>',
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                    ],
                    'S3HudiSource' => [
                        'AdditionalHudiOptions' => ['<string>', ...],
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                            'EnableSamplePath' => true || false,
                            'SamplePath' => '<string>',
                        ],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Paths' => ['<string>', ...],
                    ],
                    'S3HyperDirectTarget' => [
                        'Compression' => 'uncompressed',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Path' => '<string>',
                        'SchemaChangePolicy' => [
                            'Database' => '<string>',
                            'EnableUpdateCatalog' => true || false,
                            'Table' => '<string>',
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                    ],
                    'S3IcebergDirectTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'Compression' => 'gzip|lzo|uncompressed|snappy',
                        'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'NumberTargetPartitions' => '<string>',
                        'PartitionKeys' => [
                            ['<string>', ...],
                            // ...
                        ],
                        'Path' => '<string>',
                        'SchemaChangePolicy' => [
                            'Database' => '<string>',
                            'EnableUpdateCatalog' => true || false,
                            'Table' => '<string>',
                            'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                        ],
                    ],
                    'S3JsonSource' => [
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                            'EnableSamplePath' => true || false,
                            'SamplePath' => '<string>',
                        ],
                        'CompressionType' => 'gzip|bzip2',
                        'Exclusions' => ['<string>', ...],
                        'GroupFiles' => '<string>',
                        'GroupSize' => '<string>',
                        'JsonPath' => '<string>',
                        'MaxBand' => <integer>,
                        'MaxFilesInBand' => <integer>,
                        'Multiline' => true || false,
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Paths' => ['<string>', ...],
                        'Recurse' => true || false,
                    ],
                    'S3ParquetSource' => [
                        'AdditionalOptions' => [
                            'BoundedFiles' => <integer>,
                            'BoundedSize' => <integer>,
                            'EnableSamplePath' => true || false,
                            'SamplePath' => '<string>',
                        ],
                        'CompressionType' => 'snappy|lzo|gzip|brotli|lz4|uncompressed|none',
                        'Exclusions' => ['<string>', ...],
                        'GroupFiles' => '<string>',
                        'GroupSize' => '<string>',
                        'MaxBand' => <integer>,
                        'MaxFilesInBand' => <integer>,
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'Paths' => ['<string>', ...],
                        'Recurse' => true || false,
                    ],
                    'SelectFields' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Paths' => [
                            ['<string>', ...],
                            // ...
                        ],
                    ],
                    'SelectFromCollection' => [
                        'Index' => <integer>,
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'SnowflakeSource' => [
                        'Data' => [
                            'Action' => '<string>',
                            'AdditionalOptions' => ['<string>', ...],
                            'AutoPushdown' => true || false,
                            'Connection' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'Database' => '<string>',
                            'IamRole' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'MergeAction' => '<string>',
                            'MergeClause' => '<string>',
                            'MergeWhenMatched' => '<string>',
                            'MergeWhenNotMatched' => '<string>',
                            'PostAction' => '<string>',
                            'PreAction' => '<string>',
                            'SampleQuery' => '<string>',
                            'Schema' => '<string>',
                            'SelectedColumns' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'SourceType' => '<string>',
                            'StagingTable' => '<string>',
                            'Table' => '<string>',
                            'TableSchema' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'TempDir' => '<string>',
                            'Upsert' => true || false,
                        ],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'SnowflakeTarget' => [
                        'Data' => [
                            'Action' => '<string>',
                            'AdditionalOptions' => ['<string>', ...],
                            'AutoPushdown' => true || false,
                            'Connection' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'Database' => '<string>',
                            'IamRole' => [
                                'Description' => '<string>',
                                'Label' => '<string>',
                                'Value' => '<string>',
                            ],
                            'MergeAction' => '<string>',
                            'MergeClause' => '<string>',
                            'MergeWhenMatched' => '<string>',
                            'MergeWhenNotMatched' => '<string>',
                            'PostAction' => '<string>',
                            'PreAction' => '<string>',
                            'SampleQuery' => '<string>',
                            'Schema' => '<string>',
                            'SelectedColumns' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'SourceType' => '<string>',
                            'StagingTable' => '<string>',
                            'Table' => '<string>',
                            'TableSchema' => [
                                [
                                    'Description' => '<string>',
                                    'Label' => '<string>',
                                    'Value' => '<string>',
                                ],
                                // ...
                            ],
                            'TempDir' => '<string>',
                            'Upsert' => true || false,
                        ],
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                    ],
                    'SparkConnectorSource' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'ConnectionName' => '<string>',
                        'ConnectionType' => '<string>',
                        'ConnectorName' => '<string>',
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'SparkConnectorTarget' => [
                        'AdditionalOptions' => ['<string>', ...],
                        'ConnectionName' => '<string>',
                        'ConnectionType' => '<string>',
                        'ConnectorName' => '<string>',
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                    ],
                    'SparkSQL' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'OutputSchemas' => [
                            [
                                'Columns' => [
                                    [
                                        'Name' => '<string>',
                                        'Type' => '<string>',
                                    ],
                                    // ...
                                ],
                            ],
                            // ...
                        ],
                        'SqlAliases' => [
                            [
                                'Alias' => '<string>',
                                'From' => '<string>',
                            ],
                            // ...
                        ],
                        'SqlQuery' => '<string>',
                    ],
                    'Spigot' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Path' => '<string>',
                        'Prob' => <float>,
                        'Topk' => <integer>,
                    ],
                    'SplitFields' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'Paths' => [
                            ['<string>', ...],
                            // ...
                        ],
                    ],
                    'Union' => [
                        'Inputs' => ['<string>', ...],
                        'Name' => '<string>',
                        'UnionType' => 'ALL|DISTINCT',
                    ],
                ],
                // ...
            ],
            'Command' => [
                'Name' => '<string>',
                'PythonVersion' => '<string>',
                'Runtime' => '<string>',
                'ScriptLocation' => '<string>',
            ],
            'Connections' => [
                'Connections' => ['<string>', ...],
            ],
            'CreatedOn' => <DateTime>,
            'DefaultArguments' => ['<string>', ...],
            'Description' => '<string>',
            'ExecutionClass' => 'FLEX|STANDARD',
            'ExecutionProperty' => [
                'MaxConcurrentRuns' => <integer>,
            ],
            'GlueVersion' => '<string>',
            'JobMode' => 'SCRIPT|VISUAL|NOTEBOOK',
            'JobRunQueuingEnabled' => true || false,
            'LastModifiedOn' => <DateTime>,
            'LogUri' => '<string>',
            'MaintenanceWindow' => '<string>',
            'MaxCapacity' => <float>,
            'MaxRetries' => <integer>,
            'Name' => '<string>',
            'NonOverridableArguments' => ['<string>', ...],
            'NotificationProperty' => [
                'NotifyDelayAfter' => <integer>,
            ],
            'NumberOfWorkers' => <integer>,
            'ProfileName' => '<string>',
            'Role' => '<string>',
            'SecurityConfiguration' => '<string>',
            'SourceControlDetails' => [
                'AuthStrategy' => 'PERSONAL_ACCESS_TOKEN|AWS_SECRETS_MANAGER',
                'AuthToken' => '<string>',
                'Branch' => '<string>',
                'Folder' => '<string>',
                'LastCommitId' => '<string>',
                'Owner' => '<string>',
                'Provider' => 'GITHUB|GITLAB|BITBUCKET|AWS_CODE_COMMIT',
                'Repository' => '<string>',
            ],
            'Timeout' => <integer>,
            'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
        ],
        // ...
    ],
    'JobsNotFound' => ['<string>', ...],
]

Result Details

Errors

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

BatchGetPartition

$result = $client->batchGetPartition([/* ... */]);
$promise = $client->batchGetPartitionAsync([/* ... */]);

Retrieves partitions in a batch request.

Parameter Syntax

$result = $client->batchGetPartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionsToGet' => [ // REQUIRED
        [
            'Values' => ['<string>', ...], // REQUIRED
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Partitions' => [
        [
            'CatalogId' => '<string>',
            'CreationTime' => <DateTime>,
            'DatabaseName' => '<string>',
            'LastAccessTime' => <DateTime>,
            'LastAnalyzedTime' => <DateTime>,
            'Parameters' => ['<string>', ...],
            'StorageDescriptor' => [
                'AdditionalLocations' => ['<string>', ...],
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>',
                        'Parameters' => ['<string>', ...],
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SchemaReference' => [
                    'SchemaId' => [
                        'RegistryName' => '<string>',
                        'SchemaArn' => '<string>',
                        'SchemaName' => '<string>',
                    ],
                    'SchemaVersionId' => '<string>',
                    'SchemaVersionNumber' => <integer>,
                ],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>',
                        'SortOrder' => <integer>,
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'TableName' => '<string>',
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
    'UnprocessedKeys' => [
        [
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

EntityNotFoundException:

A specified entity does not exist

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

GlueEncryptionException:

An encryption operation failed.

InvalidStateException:

An error that indicates your data is in an invalid state.

FederationSourceException:

A federation source failed.

FederationSourceRetryableException:

A federation source failed, but the operation may be retried.

BatchGetTableOptimizer

$result = $client->batchGetTableOptimizer([/* ... */]);
$promise = $client->batchGetTableOptimizerAsync([/* ... */]);

Returns the configuration for the specified table optimizers.

Parameter Syntax

$result = $client->batchGetTableOptimizer([
    'Entries' => [ // REQUIRED
        [
            'catalogId' => '<string>',
            'databaseName' => '<string>',
            'tableName' => '<string>',
            'type' => 'compaction|retention|orphan_file_deletion',
        ],
        // ...
    ],
]);

Parameter Details

Result Syntax

[
    'Failures' => [
        [
            'catalogId' => '<string>',
            'databaseName' => '<string>',
            'error' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'tableName' => '<string>',
            'type' => 'compaction|retention|orphan_file_deletion',
        ],
        // ...
    ],
    'TableOptimizers' => [
        [
            'catalogId' => '<string>',
            'databaseName' => '<string>',
            'tableName' => '<string>',
            'tableOptimizer' => [
                'configuration' => [
                    'enabled' => true || false,
                    'orphanFileDeletionConfiguration' => [
                        'icebergConfiguration' => [
                            'location' => '<string>',
                            'orphanFileRetentionPeriodInDays' => <integer>,
                        ],
                    ],
                    'retentionConfiguration' => [
                        'icebergConfiguration' => [
                            'cleanExpiredFiles' => true || false,
                            'numberOfSnapshotsToRetain' => <integer>,
                            'snapshotRetentionPeriodInDays' => <integer>,
                        ],
                    ],
                    'roleArn' => '<string>',
                    'vpcConfiguration' => [
                        'glueConnectionName' => '<string>',
                    ],
                ],
                'lastRun' => [
                    'compactionMetrics' => [
                        'IcebergMetrics' => [
                            'DpuHours' => <float>,
                            'JobDurationInHour' => <float>,
                            'NumberOfBytesCompacted' => <integer>,
                            'NumberOfDpus' => <integer>,
                            'NumberOfFilesCompacted' => <integer>,
                        ],
                    ],
                    'endTimestamp' => <DateTime>,
                    'error' => '<string>',
                    'eventType' => 'starting|completed|failed|in_progress',
                    'metrics' => [
                        'JobDurationInHour' => '<string>',
                        'NumberOfBytesCompacted' => '<string>',
                        'NumberOfDpus' => '<string>',
                        'NumberOfFilesCompacted' => '<string>',
                    ],
                    'orphanFileDeletionMetrics' => [
                        'IcebergMetrics' => [
                            'DpuHours' => <float>,
                            'JobDurationInHour' => <float>,
                            'NumberOfDpus' => <integer>,
                            'NumberOfOrphanFilesDeleted' => <integer>,
                        ],
                    ],
                    'retentionMetrics' => [
                        'IcebergMetrics' => [
                            'DpuHours' => <float>,
                            'JobDurationInHour' => <float>,
                            'NumberOfDataFilesDeleted' => <integer>,
                            'NumberOfDpus' => <integer>,
                            'NumberOfManifestFilesDeleted' => <integer>,
                            'NumberOfManifestListsDeleted' => <integer>,
                        ],
                    ],
                    'startTimestamp' => <DateTime>,
                ],
                'type' => 'compaction|retention|orphan_file_deletion',
            ],
        ],
        // ...
    ],
]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

AccessDeniedException:

Access to a resource was denied.

InternalServiceException:

An internal service error occurred.

ThrottlingException:

The throttling threshhold was exceeded.

BatchGetTriggers

$result = $client->batchGetTriggers([/* ... */]);
$promise = $client->batchGetTriggersAsync([/* ... */]);

Returns a list of resource metadata for a given list of trigger names. After calling the ListTriggers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetTriggers([
    'TriggerNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Triggers' => [
        [
            'Actions' => [
                [
                    'Arguments' => ['<string>', ...],
                    'CrawlerName' => '<string>',
                    'JobName' => '<string>',
                    'NotificationProperty' => [
                        'NotifyDelayAfter' => <integer>,
                    ],
                    'SecurityConfiguration' => '<string>',
                    'Timeout' => <integer>,
                ],
                // ...
            ],
            'Description' => '<string>',
            'EventBatchingCondition' => [
                'BatchSize' => <integer>,
                'BatchWindow' => <integer>,
            ],
            'Id' => '<string>',
            'Name' => '<string>',
            'Predicate' => [
                'Conditions' => [
                    [
                        'CrawlState' => 'RUNNING|CANCELLING|CANCELLED|SUCCEEDED|FAILED|ERROR',
                        'CrawlerName' => '<string>',
                        'JobName' => '<string>',
                        'LogicalOperator' => 'EQUALS',
                        'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT|ERROR|WAITING|EXPIRED',
                    ],
                    // ...
                ],
                'Logical' => 'AND|ANY',
            ],
            'Schedule' => '<string>',
            'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
            'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND|EVENT',
            'WorkflowName' => '<string>',
        ],
        // ...
    ],
    'TriggersNotFound' => ['<string>', ...],
]

Result Details

Errors

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

BatchGetWorkflows

$result = $client->batchGetWorkflows([/* ... */]);
$promise = $client->batchGetWorkflowsAsync([/* ... */]);

Returns a list of resource metadata for a given list of workflow names. After calling the ListWorkflows operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetWorkflows([
    'IncludeGraph' => true || false,
    'Names' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'MissingWorkflows' => ['<string>', ...],
    'Workflows' => [
        [
            'BlueprintDetails' => [
                'BlueprintName' => '<string>',
                'RunId' => '<string>',
            ],
            'CreatedOn' => <DateTime>,
            'DefaultRunProperties' => ['<string>', ...],
            'Description' => '<string>',
            'Graph' => [
                'Edges' => [
                    [
                        'DestinationId' => '<string>',
                        'SourceId' => '<string>',
                    ],
                    // ...
                ],
                'Nodes' => [
                    [
                        'CrawlerDetails' => [
                            'Crawls' => [
                                [
                                    'CompletedOn' => <DateTime>,
                                    'ErrorMessage' => '<string>',
                                    'LogGroup' => '<string>',
                                    'LogStream' => '<string>',
                                    'StartedOn' => <DateTime>,
                                    'State' => 'RUNNING|CANCELLING|CANCELLED|SUCCEEDED|FAILED|ERROR',
                                ],
                                // ...
                            ],
                        ],
                        'JobDetails' => [
                            'JobRuns' => [
                                [
                                    'AllocatedCapacity' => <integer>,
                                    'Arguments' => ['<string>', ...],
                                    'Attempt' => <integer>,
                                    'CompletedOn' => <DateTime>,
                                    'DPUSeconds' => <float>,
                                    'ErrorMessage' => '<string>',
                                    'ExecutionClass' => 'FLEX|STANDARD',
                                    'ExecutionTime' => <integer>,
                                    'GlueVersion' => '<string>',
                                    'Id' => '<string>',
                                    'JobMode' => 'SCRIPT|VISUAL|NOTEBOOK',
                                    'JobName' => '<string>',
                                    'JobRunQueuingEnabled' => true || false,
                                    'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT|ERROR|WAITING|EXPIRED',
                                    'LastModifiedOn' => <DateTime>,
                                    'LogGroupName' => '<string>',
                                    'MaintenanceWindow' => '<string>',
                                    'MaxCapacity' => <float>,
                                    'NotificationProperty' => [
                                        'NotifyDelayAfter' => <integer>,
                                    ],
                                    'NumberOfWorkers' => <integer>,
                                    'PredecessorRuns' => [
                                        [
                                            'JobName' => '<string>',
                                            'RunId' => '<string>',
                                        ],
                                        // ...
                                    ],
                                    'PreviousRunId' => '<string>',
                                    'ProfileName' => '<string>',
                                    'SecurityConfiguration' => '<string>',
                                    'StartedOn' => <DateTime>,
                                    'StateDetail' => '<string>',
                                    'Timeout' => <integer>,
                                    'TriggerName' => '<string>',
                                    'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
                                ],
                                // ...
                            ],
                        ],
                        'Name' => '<string>',
                        'TriggerDetails' => [
                            'Trigger' => [
                                'Actions' => [
                                    [
                                        'Arguments' => ['<string>', ...],
                                        'CrawlerName' => '<string>',
                                        'JobName' => '<string>',
                                        'NotificationProperty' => [
                                            'NotifyDelayAfter' => <integer>,
                                        ],
                                        'SecurityConfiguration' => '<string>',
                                        'Timeout' => <integer>,
                                    ],
                                    // ...
                                ],
                                'Description' => '<string>',
                                'EventBatchingCondition' => [
                                    'BatchSize' => <integer>,
                                    'BatchWindow' => <integer>,
                                ],
                                'Id' => '<string>',
                                'Name' => '<string>',
                                'Predicate' => [
                                    'Conditions' => [
                                        [
                                            'CrawlState' => 'RUNNING|CANCELLING|CANCELLED|SUCCEEDED|FAILED|ERROR',
                                            'CrawlerName' => '<string>',
                                            'JobName' => '<string>',
                                            'LogicalOperator' => 'EQUALS',
                                            'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT|ERROR|WAITING|EXPIRED',
                                        ],
                                        // ...
                                    ],
                                    'Logical' => 'AND|ANY',
                                ],
                                'Schedule' => '<string>',
                                'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
                                'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND|EVENT',
                                'WorkflowName' => '<string>',
                            ],
                        ],
                        'Type' => 'CRAWLER|JOB|TRIGGER',
                        'UniqueId' => '<string>',
                    ],
                    // ...
                ],
            ],
            'LastModifiedOn' => <DateTime>,
            'LastRun' => [
                'CompletedOn' => <DateTime>,
                'ErrorMessage' => '<string>',
                'Graph' => [
                    'Edges' => [
                        [
                            'DestinationId' => '<string>',
                            'SourceId' => '<string>',
                        ],
                        // ...
                    ],
                    'Nodes' => [
                        [
                            'CrawlerDetails' => [
                                'Crawls' => [
                                    [
                                        'CompletedOn' => <DateTime>,
                                        'ErrorMessage' => '<string>',
                                        'LogGroup' => '<string>',
                                        'LogStream' => '<string>',
                                        'StartedOn' => <DateTime>,
                                        'State' => 'RUNNING|CANCELLING|CANCELLED|SUCCEEDED|FAILED|ERROR',
                                    ],
                                    // ...
                                ],
                            ],
                            'JobDetails' => [
                                'JobRuns' => [
                                    [
                                        'AllocatedCapacity' => <integer>,
                                        'Arguments' => ['<string>', ...],
                                        'Attempt' => <integer>,
                                        'CompletedOn' => <DateTime>,
                                        'DPUSeconds' => <float>,
                                        'ErrorMessage' => '<string>',
                                        'ExecutionClass' => 'FLEX|STANDARD',
                                        'ExecutionTime' => <integer>,
                                        'GlueVersion' => '<string>',
                                        'Id' => '<string>',
                                        'JobMode' => 'SCRIPT|VISUAL|NOTEBOOK',
                                        'JobName' => '<string>',
                                        'JobRunQueuingEnabled' => true || false,
                                        'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT|ERROR|WAITING|EXPIRED',
                                        'LastModifiedOn' => <DateTime>,
                                        'LogGroupName' => '<string>',
                                        'MaintenanceWindow' => '<string>',
                                        'MaxCapacity' => <float>,
                                        'NotificationProperty' => [
                                            'NotifyDelayAfter' => <integer>,
                                        ],
                                        'NumberOfWorkers' => <integer>,
                                        'PredecessorRuns' => [
                                            [
                                                'JobName' => '<string>',
                                                'RunId' => '<string>',
                                            ],
                                            // ...
                                        ],
                                        'PreviousRunId' => '<string>',
                                        'ProfileName' => '<string>',
                                        'SecurityConfiguration' => '<string>',
                                        'StartedOn' => <DateTime>,
                                        'StateDetail' => '<string>',
                                        'Timeout' => <integer>,
                                        'TriggerName' => '<string>',
                                        'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
                                    ],
                                    // ...
                                ],
                            ],
                            'Name' => '<string>',
                            'TriggerDetails' => [
                                'Trigger' => [
                                    'Actions' => [
                                        [
                                            'Arguments' => ['<string>', ...],
                                            'CrawlerName' => '<string>',
                                            'JobName' => '<string>',
                                            'NotificationProperty' => [
                                                'NotifyDelayAfter' => <integer>,
                                            ],
                                            'SecurityConfiguration' => '<string>',
                                            'Timeout' => <integer>,
                                        ],
                                        // ...
                                    ],
                                    'Description' => '<string>',
                                    'EventBatchingCondition' => [
                                        'BatchSize' => <integer>,
                                        'BatchWindow' => <integer>,
                                    ],
                                    'Id' => '<string>',
                                    'Name' => '<string>',
                                    'Predicate' => [
                                        'Conditions' => [
                                            [
                                                'CrawlState' => 'RUNNING|CANCELLING|CANCELLED|SUCCEEDED|FAILED|ERROR',
                                                'CrawlerName' => '<string>',
                                                'JobName' => '<string>',
                                                'LogicalOperator' => 'EQUALS',
                                                'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT|ERROR|WAITING|EXPIRED',
                                            ],
                                            // ...
                                        ],
                                        'Logical' => 'AND|ANY',
                                    ],
                                    'Schedule' => '<string>',
                                    'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
                                    'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND|EVENT',
                                    'WorkflowName' => '<string>',
                                ],
                            ],
                            'Type' => 'CRAWLER|JOB|TRIGGER',
                            'UniqueId' => '<string>',
                        ],
                        // ...
                    ],
                ],
                'Name' => '<string>',
                'PreviousRunId' => '<string>',
                'StartedOn' => <DateTime>,
                'StartingEventBatchCondition' => [
                    'BatchSize' => <integer>,
                    'BatchWindow' => <integer>,
                ],
                'Statistics' => [
                    'ErroredActions' => <integer>,
                    'FailedActions' => <integer>,
                    'RunningActions' => <integer>,
                    'StoppedActions' => <integer>,
                    'SucceededActions' => <integer>,
                    'TimeoutActions' => <integer>,
                    'TotalActions' => <integer>,
                    'WaitingActions' => <integer>,
                ],
                'Status' => 'RUNNING|COMPLETED|STOPPING|STOPPED|ERROR',
                'WorkflowRunId' => '<string>',
                'WorkflowRunProperties' => ['<string>', ...],
            ],
            'MaxConcurrentRuns' => <integer>,
            'Name' => '<string>',
        ],
        // ...
    ],
]

Result Details

Errors

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

BatchPutDataQualityStatisticAnnotation

$result = $client->batchPutDataQualityStatisticAnnotation([/* ... */]);
$promise = $client->batchPutDataQualityStatisticAnnotationAsync([/* ... */]);

Annotate datapoints over time for a specific data quality statistic.

Parameter Syntax

$result = $client->batchPutDataQualityStatisticAnnotation([
    'ClientToken' => '<string>',
    'InclusionAnnotations' => [ // REQUIRED
        [
            'InclusionAnnotation' => 'INCLUDE|EXCLUDE',
            'ProfileId' => '<string>',
            'StatisticId' => '<string>',
        ],
        // ...
    ],
]);

Parameter Details

Result Syntax

[
    'FailedInclusionAnnotations' => [
        [
            'FailureReason' => '<string>',
            'ProfileId' => '<string>',
            'StatisticId' => '<string>',
        ],
        // ...
    ],
]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

InternalServiceException:

An internal service error occurred.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

BatchStopJobRun

$result = $client->batchStopJobRun([/* ... */]);
$promise = $client->batchStopJobRunAsync([/* ... */]);

Stops one or more job runs for a specified job definition.

Parameter Syntax

$result = $client->batchStopJobRun([
    'JobName' => '<string>', // REQUIRED
    'JobRunIds' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'JobName' => '<string>',
            'JobRunId' => '<string>',
        ],
        // ...
    ],
    'SuccessfulSubmissions' => [
        [
            'JobName' => '<string>',
            'JobRunId' => '<string>',
        ],
        // ...
    ],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

BatchUpdatePartition

$result = $client->batchUpdatePartition([/* ... */]);
$promise = $client->batchUpdatePartitionAsync([/* ... */]);

Updates one or more partitions in a batch operation.

Parameter Syntax

$result = $client->batchUpdatePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'Entries' => [ // REQUIRED
        [
            'PartitionInput' => [ // REQUIRED
                'LastAccessTime' => <integer || string || DateTime>,
                'LastAnalyzedTime' => <integer || string || DateTime>,
                'Parameters' => ['<string>', ...],
                'StorageDescriptor' => [
                    'AdditionalLocations' => ['<string>', ...],
                    'BucketColumns' => ['<string>', ...],
                    'Columns' => [
                        [
                            'Comment' => '<string>',
                            'Name' => '<string>', // REQUIRED
                            'Parameters' => ['<string>', ...],
                            'Type' => '<string>',
                        ],
                        // ...
                    ],
                    'Compressed' => true || false,
                    'InputFormat' => '<string>',
                    'Location' => '<string>',
                    'NumberOfBuckets' => <integer>,
                    'OutputFormat' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SchemaReference' => [
                        'SchemaId' => [
                            'RegistryName' => '<string>',
                            'SchemaArn' => '<string>',
                            'SchemaName' => '<string>',
                        ],
                        'SchemaVersionId' => '<string>',
                        'SchemaVersionNumber' => <integer>,
                    ],
                    'SerdeInfo' => [
                        'Name' => '<string>',
                        'Parameters' => ['<string>', ...],
                        'SerializationLibrary' => '<string>',
                    ],
                    'SkewedInfo' => [
                        'SkewedColumnNames' => ['<string>', ...],
                        'SkewedColumnValueLocationMaps' => ['<string>', ...],
                        'SkewedColumnValues' => ['<string>', ...],
                    ],
                    'SortColumns' => [
                        [
                            'Column' => '<string>', // REQUIRED
                            'SortOrder' => <integer>, // REQUIRED
                        ],
                        // ...
                    ],
                    'StoredAsSubDirectories' => true || false,
                ],
                'Values' => ['<string>', ...],
            ],
            'PartitionValueList' => ['<string>', ...], // REQUIRED
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'PartitionValueList' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

EntityNotFoundException:

A specified entity does not exist

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

GlueEncryptionException:

An encryption operation failed.

CancelDataQualityRuleRecommendationRun

$result = $client->cancelDataQualityRuleRecommendationRun([/* ... */]);
$promise = $client->cancelDataQualityRuleRecommendationRunAsync([/* ... */]);

Cancels the specified recommendation run that was being used to generate rules.

Parameter Syntax

$result = $client->cancelDataQualityRuleRecommendationRun([
    'RunId' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

CancelDataQualityRulesetEvaluationRun

$result = $client->cancelDataQualityRulesetEvaluationRun([/* ... */]);
$promise = $client->cancelDataQualityRulesetEvaluationRunAsync([/* ... */]);

Cancels a run where a ruleset is being evaluated against a data source.

Parameter Syntax

$result = $client->cancelDataQualityRulesetEvaluationRun([
    'RunId' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

CancelMLTaskRun

$result = $client->cancelMLTaskRun([/* ... */]);
$promise = $client->cancelMLTaskRunAsync([/* ... */]);

Cancels (stops) a task run. Machine learning task runs are asynchronous tasks that Glue runs on your behalf as part of various machine learning workflows. You can cancel a machine learning task run at any time by calling CancelMLTaskRun with a task run's parent transform's TransformID and the task run's TaskRunId.

Parameter Syntax

$result = $client->cancelMLTaskRun([
    'TaskRunId' => '<string>', // REQUIRED
    'TransformId' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Status' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
    'TaskRunId' => '<string>',
    'TransformId' => '<string>',
]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

CancelStatement

$result = $client->cancelStatement([/* ... */]);
$promise = $client->cancelStatementAsync([/* ... */]);

Cancels the statement.

Parameter Syntax

$result = $client->cancelStatement([
    'Id' => <integer>, // REQUIRED
    'RequestOrigin' => '<string>',
    'SessionId' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

AccessDeniedException:

Access to a resource was denied.

EntityNotFoundException:

A specified entity does not exist

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

IllegalSessionStateException:

The session is in an invalid state to perform a requested operation.

CheckSchemaVersionValidity

$result = $client->checkSchemaVersionValidity([/* ... */]);
$promise = $client->checkSchemaVersionValidityAsync([/* ... */]);

Validates the supplied schema. This call has no side effects, it simply validates using the supplied schema using DataFormat as the format. Since it does not take a schema set name, no compatibility checks are performed.

Parameter Syntax

$result = $client->checkSchemaVersionValidity([
    'DataFormat' => 'AVRO|JSON|PROTOBUF', // REQUIRED
    'SchemaDefinition' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'Error' => '<string>',
    'Valid' => true || false,
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AccessDeniedException:

Access to a resource was denied.

InternalServiceException:

An internal service error occurred.

CreateBlueprint

$result = $client->createBlueprint([/* ... */]);
$promise = $client->createBlueprintAsync([/* ... */]);

Registers a blueprint with Glue.

Parameter Syntax

$result = $client->createBlueprint([
    'BlueprintLocation' => '<string>', // REQUIRED
    'Description' => '<string>',
    'Name' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateCatalog

$result = $client->createCatalog([/* ... */]);
$promise = $client->createCatalogAsync([/* ... */]);

Creates a new catalog in the Glue Data Catalog.

Parameter Syntax

$result = $client->createCatalog([
    'CatalogInput' => [ // REQUIRED
        'AllowFullTableExternalDataAccess' => 'True|False',
        'CatalogProperties' => [
            'CustomProperties' => ['<string>', ...],
            'DataLakeAccessProperties' => [
                'CatalogType' => '<string>',
                'DataLakeAccess' => true || false,
                'DataTransferRole' => '<string>',
                'KmsKey' => '<string>',
            ],
        ],
        'CreateDatabaseDefaultPermissions' => [
            [
                'Permissions' => ['<string>', ...],
                'Principal' => [
                    'DataLakePrincipalIdentifier' => '<string>',
                ],
            ],
            // ...
        ],
        'CreateTableDefaultPermissions' => [
            [
                'Permissions' => ['<string>', ...],
                'Principal' => [
                    'DataLakePrincipalIdentifier' => '<string>',
                ],
            ],
            // ...
        ],
        'Description' => '<string>',
        'FederatedCatalog' => [
            'ConnectionName' => '<string>',
            'Identifier' => '<string>',
        ],
        'Parameters' => ['<string>', ...],
        'TargetRedshiftCatalog' => [
            'CatalogArn' => '<string>', // REQUIRED
        ],
    ],
    'Name' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

ConcurrentModificationException:

Two processes are trying to modify a resource simultaneously.

AccessDeniedException:

Access to a resource was denied.

EntityNotFoundException:

A specified entity does not exist

FederatedResourceAlreadyExistsException:

A federated resource already exists.

FederationSourceException:

A federation source failed.

CreateClassifier

$result = $client->createClassifier([/* ... */]);
$promise = $client->createClassifierAsync([/* ... */]);

Creates a classifier in the user's account. This can be a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field of the request is present.

Parameter Syntax

$result = $client->createClassifier([
    'CsvClassifier' => [
        'AllowSingleColumn' => true || false,
        'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
        'CustomDatatypeConfigured' => true || false,
        'CustomDatatypes' => ['<string>', ...],
        'Delimiter' => '<string>',
        'DisableValueTrimming' => true || false,
        'Header' => ['<string>', ...],
        'Name' => '<string>', // REQUIRED
        'QuoteSymbol' => '<string>',
        'Serde' => 'OpenCSVSerDe|LazySimpleSerDe|None',
    ],
    'GrokClassifier' => [
        'Classification' => '<string>', // REQUIRED
        'CustomPatterns' => '<string>',
        'GrokPattern' => '<string>', // REQUIRED
        'Name' => '<string>', // REQUIRED
    ],
    'JsonClassifier' => [
        'JsonPath' => '<string>', // REQUIRED
        'Name' => '<string>', // REQUIRED
    ],
    'XMLClassifier' => [
        'Classification' => '<string>', // REQUIRED
        'Name' => '<string>', // REQUIRED
        'RowTag' => '<string>',
    ],
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

CreateColumnStatisticsTaskSettings

$result = $client->createColumnStatisticsTaskSettings([/* ... */]);
$promise = $client->createColumnStatisticsTaskSettingsAsync([/* ... */]);

Creates settings for a column statistics task.

Parameter Syntax

$result = $client->createColumnStatisticsTaskSettings([
    'CatalogID' => '<string>',
    'ColumnNameList' => ['<string>', ...],
    'DatabaseName' => '<string>', // REQUIRED
    'Role' => '<string>', // REQUIRED
    'SampleSize' => <float>,
    'Schedule' => '<string>',
    'SecurityConfiguration' => '<string>',
    'TableName' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

AccessDeniedException:

Access to a resource was denied.

EntityNotFoundException:

A specified entity does not exist

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

ColumnStatisticsTaskRunningException:

An exception thrown when you try to start another job while running a column stats generation job.

CreateConnection

$result = $client->createConnection([/* ... */]);
$promise = $client->createConnectionAsync([/* ... */]);

Creates a connection definition in the Data Catalog.

Connections used for creating federated resources require the IAM glue:PassConnection permission.

Parameter Syntax

$result = $client->createConnection([
    'CatalogId' => '<string>',
    'ConnectionInput' => [ // REQUIRED
        'AthenaProperties' => ['<string>', ...],
        'AuthenticationConfiguration' => [
            'AuthenticationType' => 'BASIC|OAUTH2|CUSTOM|IAM',
            'BasicAuthenticationCredentials' => [
                'Password' => '<string>',
                'Username' => '<string>',
            ],
            'CustomAuthenticationCredentials' => ['<string>', ...],
            'KmsKeyArn' => '<string>',
            'OAuth2Properties' => [
                'AuthorizationCodeProperties' => [
                    'AuthorizationCode' => '<string>',
                    'RedirectUri' => '<string>',
                ],
                'OAuth2ClientApplication' => [
                    'AWSManagedClientApplicationReference' => '<string>',
                    'UserManagedClientApplicationClientId' => '<string>',
                ],
                'OAuth2Credentials' => [
                    'AccessToken' => '<string>',
                    'JwtToken' => '<string>',
                    'RefreshToken' => '<string>',
                    'UserManagedClientApplicationClientSecret' => '<string>',
                ],
                'OAuth2GrantType' => 'AUTHORIZATION_CODE|CLIENT_CREDENTIALS|JWT_BEARER',
                'TokenUrl' => '<string>',
                'TokenUrlParametersMap' => ['<string>', ...],
            ],
            'SecretArn' => '<string>',
        ],
        'ConnectionProperties' => ['<string>', ...], // REQUIRED
        'ConnectionType' => 'JDBC|SFTP|MONGODB|KAFKA|NETWORK|MARKETPLACE|CUSTOM|SALESFORCE|VIEW_VALIDATION_REDSHIFT|VIEW_VALIDATION_ATHENA|GOOGLEADS|GOOGLESHEETS|GOOGLEANALYTICS4|SERVICENOW|MARKETO|SAPODATA|ZENDESK|JIRACLOUD|NETSUITEERP|HUBSPOT|FACEBOOKADS|INSTAGRAMADS|ZOHOCRM|SALESFORCEPARDOT|SALESFORCEMARKETINGCLOUD|SLACK|STRIPE|INTERCOM|SNAPCHATADS', // REQUIRED
        'Description' => '<string>',
        'MatchCriteria' => ['<string>', ...],
        'Name' => '<string>', // REQUIRED
        'PhysicalConnectionRequirements' => [
            'AvailabilityZone' => '<string>',
            'SecurityGroupIdList' => ['<string>', ...],
            'SubnetId' => '<string>',
        ],
        'PythonProperties' => ['<string>', ...],
        'SparkProperties' => ['<string>', ...],
        'ValidateCredentials' => true || false,
        'ValidateForComputeEnvironments' => ['<string>', ...],
    ],
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[
    'CreateConnectionStatus' => 'READY|IN_PROGRESS|FAILED',
]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

GlueEncryptionException:

An encryption operation failed.

CreateCrawler

$result = $client->createCrawler([/* ... */]);
$promise = $client->createCrawlerAsync([/* ... */]);

Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in the s3Targets field, the jdbcTargets field, or the DynamoDBTargets field.

Parameter Syntax

$result = $client->createCrawler([
    'Classifiers' => ['<string>', ...],
    'Configuration' => '<string>',
    'CrawlerSecurityConfiguration' => '<string>',
    'DatabaseName' => '<string>',
    'Description' => '<string>',
    'LakeFormationConfiguration' => [
        'AccountId' => '<string>',
        'UseLakeFormationCredentials' => true || false,
    ],
    'LineageConfiguration' => [
        'CrawlerLineageSettings' => 'ENABLE|DISABLE',
    ],
    'Name' => '<string>', // REQUIRED
    'RecrawlPolicy' => [
        'RecrawlBehavior' => 'CRAWL_EVERYTHING|CRAWL_NEW_FOLDERS_ONLY|CRAWL_EVENT_MODE',
    ],
    'Role' => '<string>', // REQUIRED
    'Schedule' => '<string>',
    'SchemaChangePolicy' => [
        'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
        'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
    ],
    'TablePrefix' => '<string>',
    'Tags' => ['<string>', ...],
    'Targets' => [ // REQUIRED
        'CatalogTargets' => [
            [
                'ConnectionName' => '<string>',
                'DatabaseName' => '<string>', // REQUIRED
                'DlqEventQueueArn' => '<string>',
                'EventQueueArn' => '<string>',
                'Tables' => ['<string>', ...], // REQUIRED
            ],
            // ...
        ],
        'DeltaTargets' => [
            [
                'ConnectionName' => '<string>',
                'CreateNativeDeltaTable' => true || false,
                'DeltaTables' => ['<string>', ...],
                'WriteManifest' => true || false,
            ],
            // ...
        ],
        'DynamoDBTargets' => [
            [
                'Path' => '<string>',
                'scanAll' => true || false,
                'scanRate' => <float>,
            ],
            // ...
        ],
        'HudiTargets' => [
            [
                'ConnectionName' => '<string>',
                'Exclusions' => ['<string>', ...],
                'MaximumTraversalDepth' => <integer>,
                'Paths' => ['<string>', ...],
            ],
            // ...
        ],
        'IcebergTargets' => [
            [
                'ConnectionName' => '<string>',
                'Exclusions' => ['<string>', ...],
                'MaximumTraversalDepth' => <integer>,
                'Paths' => ['<string>', ...],
            ],
            // ...
        ],
        'JdbcTargets' => [
            [
                'ConnectionName' => '<string>',
                'EnableAdditionalMetadata' => ['<string>', ...],
                'Exclusions' => ['<string>', ...],
                'Path' => '<string>',
            ],
            // ...
        ],
        'MongoDBTargets' => [
            [
                'ConnectionName' => '<string>',
                'Path' => '<string>',
                'ScanAll' => true || false,
            ],
            // ...
        ],
        'S3Targets' => [
            [
                'ConnectionName' => '<string>',
                'DlqEventQueueArn' => '<string>',
                'EventQueueArn' => '<string>',
                'Exclusions' => ['<string>', ...],
                'Path' => '<string>',
                'SampleSize' => <integer>,
            ],
            // ...
        ],
    ],
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AlreadyExistsException:

A resource to be created or added already exists.

OperationTimeoutException:

The operation timed out.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateCustomEntityType

$result = $client->createCustomEntityType([/* ... */]);
$promise = $client->createCustomEntityTypeAsync([/* ... */]);

Creates a custom pattern that is used to detect sensitive data across the columns and rows of your structured data.

Each custom pattern you create specifies a regular expression and an optional list of context words. If no context words are passed only a regular expression is checked.

Parameter Syntax

$result = $client->createCustomEntityType([
    'ContextWords' => ['<string>', ...],
    'Name' => '<string>', // REQUIRED
    'RegexString' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Errors

AccessDeniedException:

Access to a resource was denied.

AlreadyExistsException:

A resource to be created or added already exists.

IdempotentParameterMismatchException:

The same unique identifier was associated with two different records.

InternalServiceException:

An internal service error occurred.

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateDataQualityRuleset

$result = $client->createDataQualityRuleset([/* ... */]);
$promise = $client->createDataQualityRulesetAsync([/* ... */]);

Creates a data quality ruleset with DQDL rules applied to a specified Glue table.

You create the ruleset using the Data Quality Definition Language (DQDL). For more information, see the Glue developer guide.

Parameter Syntax

$result = $client->createDataQualityRuleset([
    'ClientToken' => '<string>',
    'DataQualitySecurityConfiguration' => '<string>',
    'Description' => '<string>',
    'Name' => '<string>', // REQUIRED
    'Ruleset' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
    'TargetTable' => [
        'CatalogId' => '<string>',
        'DatabaseName' => '<string>', // REQUIRED
        'TableName' => '<string>', // REQUIRED
    ],
]);

Parameter Details

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AlreadyExistsException:

A resource to be created or added already exists.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateDatabase

$result = $client->createDatabase([/* ... */]);
$promise = $client->createDatabaseAsync([/* ... */]);

Creates a new database in a Data Catalog.

Parameter Syntax

$result = $client->createDatabase([
    'CatalogId' => '<string>',
    'DatabaseInput' => [ // REQUIRED
        'CreateTableDefaultPermissions' => [
            [
                'Permissions' => ['<string>', ...],
                'Principal' => [
                    'DataLakePrincipalIdentifier' => '<string>',
                ],
            ],
            // ...
        ],
        'Description' => '<string>',
        'FederatedDatabase' => [
            'ConnectionName' => '<string>',
            'Identifier' => '<string>',
        ],
        'LocationUri' => '<string>',
        'Name' => '<string>', // REQUIRED
        'Parameters' => ['<string>', ...],
        'TargetDatabase' => [
            'CatalogId' => '<string>',
            'DatabaseName' => '<string>',
            'Region' => '<string>',
        ],
    ],
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

ConcurrentModificationException:

Two processes are trying to modify a resource simultaneously.

FederatedResourceAlreadyExistsException:

A federated resource already exists.

FederationSourceException:

A federation source failed.

FederationSourceRetryableException:

A federation source failed, but the operation may be retried.

CreateDevEndpoint

$result = $client->createDevEndpoint([/* ... */]);
$promise = $client->createDevEndpointAsync([/* ... */]);

Creates a new development endpoint.

Parameter Syntax

$result = $client->createDevEndpoint([
    'Arguments' => ['<string>', ...],
    'EndpointName' => '<string>', // REQUIRED
    'ExtraJarsS3Path' => '<string>',
    'ExtraPythonLibsS3Path' => '<string>',
    'GlueVersion' => '<string>',
    'NumberOfNodes' => <integer>,
    'NumberOfWorkers' => <integer>,
    'PublicKey' => '<string>',
    'PublicKeys' => ['<string>', ...],
    'RoleArn' => '<string>', // REQUIRED
    'SecurityConfiguration' => '<string>',
    'SecurityGroupIds' => ['<string>', ...],
    'SubnetId' => '<string>',
    'Tags' => ['<string>', ...],
    'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
]);

Parameter Details

Result Syntax

[
    'Arguments' => ['<string>', ...],
    'AvailabilityZone' => '<string>',
    'CreatedTimestamp' => <DateTime>,
    'EndpointName' => '<string>',
    'ExtraJarsS3Path' => '<string>',
    'ExtraPythonLibsS3Path' => '<string>',
    'FailureReason' => '<string>',
    'GlueVersion' => '<string>',
    'NumberOfNodes' => <integer>,
    'NumberOfWorkers' => <integer>,
    'RoleArn' => '<string>',
    'SecurityConfiguration' => '<string>',
    'SecurityGroupIds' => ['<string>', ...],
    'Status' => '<string>',
    'SubnetId' => '<string>',
    'VpcId' => '<string>',
    'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
    'YarnEndpointAddress' => '<string>',
    'ZeppelinRemoteSparkInterpreterPort' => <integer>,
]

Result Details

Errors

AccessDeniedException:

Access to a resource was denied.

AlreadyExistsException:

A resource to be created or added already exists.

IdempotentParameterMismatchException:

The same unique identifier was associated with two different records.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

ValidationException:

A value could not be validated.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateIntegration

$result = $client->createIntegration([/* ... */]);
$promise = $client->createIntegrationAsync([/* ... */]);

Creates a Zero-ETL integration in the caller's account between two resources with Amazon Resource Names (ARNs): the SourceArn and TargetArn.

Parameter Syntax

$result = $client->createIntegration([
    'AdditionalEncryptionContext' => ['<string>', ...],
    'DataFilter' => '<string>',
    'Description' => '<string>',
    'IntegrationConfig' => [
        'RefreshInterval' => '<string>',
    ],
    'IntegrationName' => '<string>', // REQUIRED
    'KmsKeyId' => '<string>',
    'SourceArn' => '<string>', // REQUIRED
    'Tags' => [
        [
            'key' => '<string>',
            'value' => '<string>',
        ],
        // ...
    ],
    'TargetArn' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'AdditionalEncryptionContext' => ['<string>', ...],
    'CreateTime' => <DateTime>,
    'DataFilter' => '<string>',
    'Description' => '<string>',
    'Errors' => [
        [
            'ErrorCode' => '<string>',
            'ErrorMessage' => '<string>',
        ],
        // ...
    ],
    'IntegrationArn' => '<string>',
    'IntegrationConfig' => [
        'RefreshInterval' => '<string>',
    ],
    'IntegrationName' => '<string>',
    'KmsKeyId' => '<string>',
    'SourceArn' => '<string>',
    'Status' => 'CREATING|ACTIVE|MODIFYING|FAILED|DELETING|SYNCING|NEEDS_ATTENTION',
    'Tags' => [
        [
            'key' => '<string>',
            'value' => '<string>',
        ],
        // ...
    ],
    'TargetArn' => '<string>',
]

Result Details

Errors

ValidationException:

A value could not be validated.

AccessDeniedException:

Access to a resource was denied.

ResourceNotFoundException:

The resource could not be found.

InternalServerException:

An internal server error occurred.

IntegrationConflictOperationFault:

The requested operation conflicts with another operation.

IntegrationQuotaExceededFault:

The data processed through your integration exceeded your quota.

KMSKeyNotAccessibleFault:

The KMS key specified is not accessible.

EntityNotFoundException:

A specified entity does not exist

InternalServiceException:

An internal service error occurred.

ConflictException:

The CreatePartitions API was called on a table that has indexes enabled.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InvalidInputException:

The input provided was not valid.

CreateIntegrationResourceProperty

$result = $client->createIntegrationResourceProperty([/* ... */]);
$promise = $client->createIntegrationResourcePropertyAsync([/* ... */]);

This API can be used for setting up the ResourceProperty of the Glue connection (for the source) or Glue database ARN (for the target). These properties can include the role to access the connection or database. To set both source and target properties the same API needs to be invoked with the Glue connection ARN as ResourceArn with SourceProcessingProperties and the Glue database ARN as ResourceArn with TargetProcessingProperties respectively.

Parameter Syntax

$result = $client->createIntegrationResourceProperty([
    'ResourceArn' => '<string>', // REQUIRED
    'SourceProcessingProperties' => [
        'RoleArn' => '<string>',
    ],
    'TargetProcessingProperties' => [
        'ConnectionName' => '<string>',
        'EventBusArn' => '<string>',
        'KmsArn' => '<string>',
        'RoleArn' => '<string>',
    ],
]);

Parameter Details

Result Syntax

[
    'ResourceArn' => '<string>',
    'SourceProcessingProperties' => [
        'RoleArn' => '<string>',
    ],
    'TargetProcessingProperties' => [
        'ConnectionName' => '<string>',
        'EventBusArn' => '<string>',
        'KmsArn' => '<string>',
        'RoleArn' => '<string>',
    ],
]

Result Details

Errors

ValidationException:

A value could not be validated.

AccessDeniedException:

Access to a resource was denied.

ConflictException:

The CreatePartitions API was called on a table that has indexes enabled.

InternalServerException:

An internal server error occurred.

ResourceNotFoundException:

The resource could not be found.

EntityNotFoundException:

A specified entity does not exist

InternalServiceException:

An internal service error occurred.

InvalidInputException:

The input provided was not valid.

CreateIntegrationTableProperties

$result = $client->createIntegrationTableProperties([/* ... */]);
$promise = $client->createIntegrationTablePropertiesAsync([/* ... */]);

This API is used to provide optional override properties for the the tables that need to be replicated. These properties can include properties for filtering and partitioning for the source and target tables. To set both source and target properties the same API need to be invoked with the Glue connection ARN as ResourceArn with SourceTableConfig, and the Glue database ARN as ResourceArn with TargetTableConfig respectively.

Parameter Syntax

$result = $client->createIntegrationTableProperties([
    'ResourceArn' => '<string>', // REQUIRED
    'SourceTableConfig' => [
        'Fields' => ['<string>', ...],
        'FilterPredicate' => '<string>',
        'PrimaryKey' => ['<string>', ...],
        'RecordUpdateField' => '<string>',
    ],
    'TableName' => '<string>', // REQUIRED
    'TargetTableConfig' => [
        'PartitionSpec' => [
            [
                'ConversionSpec' => '<string>',
                'FieldName' => '<string>',
                'FunctionSpec' => '<string>',
            ],
            // ...
        ],
        'TargetTableName' => '<string>',
        'UnnestSpec' => 'TOPLEVEL|FULL|NOUNNEST',
    ],
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

ValidationException:

A value could not be validated.

AccessDeniedException:

Access to a resource was denied.

ResourceNotFoundException:

The resource could not be found.

InternalServerException:

An internal server error occurred.

EntityNotFoundException:

A specified entity does not exist

InternalServiceException:

An internal service error occurred.

InvalidInputException:

The input provided was not valid.

CreateJob

$result = $client->createJob([/* ... */]);
$promise = $client->createJobAsync([/* ... */]);

Creates a new job definition.

Parameter Syntax

$result = $client->createJob([
    'AllocatedCapacity' => <integer>,
    'CodeGenConfigurationNodes' => [
        '<NodeId>' => [
            'Aggregate' => [
                'Aggs' => [ // REQUIRED
                    [
                        'AggFunc' => 'avg|countDistinct|count|first|last|kurtosis|max|min|skewness|stddev_samp|stddev_pop|sum|sumDistinct|var_samp|var_pop', // REQUIRED
                        'Column' => ['<string>', ...], // REQUIRED
                    ],
                    // ...
                ],
                'Groups' => [ // REQUIRED
                    ['<string>', ...],
                    // ...
                ],
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
            ],
            'AmazonRedshiftSource' => [
                'Data' => [
                    'AccessType' => '<string>',
                    'Action' => '<string>',
                    'AdvancedOptions' => [
                        [
                            'Key' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'CatalogDatabase' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'CatalogRedshiftSchema' => '<string>',
                    'CatalogRedshiftTable' => '<string>',
                    'CatalogTable' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'Connection' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'CrawlerConnection' => '<string>',
                    'IamRole' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'MergeAction' => '<string>',
                    'MergeClause' => '<string>',
                    'MergeWhenMatched' => '<string>',
                    'MergeWhenNotMatched' => '<string>',
                    'PostAction' => '<string>',
                    'PreAction' => '<string>',
                    'SampleQuery' => '<string>',
                    'Schema' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'SelectedColumns' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'SourceType' => '<string>',
                    'StagingTable' => '<string>',
                    'Table' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'TablePrefix' => '<string>',
                    'TableSchema' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'TempDir' => '<string>',
                    'Upsert' => true || false,
                ],
                'Name' => '<string>',
            ],
            'AmazonRedshiftTarget' => [
                'Data' => [
                    'AccessType' => '<string>',
                    'Action' => '<string>',
                    'AdvancedOptions' => [
                        [
                            'Key' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'CatalogDatabase' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'CatalogRedshiftSchema' => '<string>',
                    'CatalogRedshiftTable' => '<string>',
                    'CatalogTable' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'Connection' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'CrawlerConnection' => '<string>',
                    'IamRole' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'MergeAction' => '<string>',
                    'MergeClause' => '<string>',
                    'MergeWhenMatched' => '<string>',
                    'MergeWhenNotMatched' => '<string>',
                    'PostAction' => '<string>',
                    'PreAction' => '<string>',
                    'SampleQuery' => '<string>',
                    'Schema' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'SelectedColumns' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'SourceType' => '<string>',
                    'StagingTable' => '<string>',
                    'Table' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'TablePrefix' => '<string>',
                    'TableSchema' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'TempDir' => '<string>',
                    'Upsert' => true || false,
                ],
                'Inputs' => ['<string>', ...],
                'Name' => '<string>',
            ],
            'ApplyMapping' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Mapping' => [ // REQUIRED
                    [
                        'Children' => [...], // RECURSIVE
                        'Dropped' => true || false,
                        'FromPath' => ['<string>', ...],
                        'FromType' => '<string>',
                        'ToKey' => '<string>',
                        'ToType' => '<string>',
                    ],
                    // ...
                ],
                'Name' => '<string>', // REQUIRED
            ],
            'AthenaConnectorSource' => [
                'ConnectionName' => '<string>', // REQUIRED
                'ConnectionTable' => '<string>',
                'ConnectionType' => '<string>', // REQUIRED
                'ConnectorName' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'SchemaName' => '<string>', // REQUIRED
            ],
            'CatalogDeltaSource' => [
                'AdditionalDeltaOptions' => ['<string>', ...],
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'CatalogHudiSource' => [
                'AdditionalHudiOptions' => ['<string>', ...],
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'CatalogKafkaSource' => [
                'DataPreviewOptions' => [
                    'PollingTime' => <integer>,
                    'RecordPollingLimit' => <integer>,
                ],
                'Database' => '<string>', // REQUIRED
                'DetectSchema' => true || false,
                'Name' => '<string>', // REQUIRED
                'StreamingOptions' => [
                    'AddRecordTimestamp' => '<string>',
                    'Assign' => '<string>',
                    'BootstrapServers' => '<string>',
                    'Classification' => '<string>',
                    'ConnectionName' => '<string>',
                    'Delimiter' => '<string>',
                    'EmitConsumerLagMetrics' => '<string>',
                    'EndingOffsets' => '<string>',
                    'IncludeHeaders' => true || false,
                    'MaxOffsetsPerTrigger' => <integer>,
                    'MinPartitions' => <integer>,
                    'NumRetries' => <integer>,
                    'PollTimeoutMs' => <integer>,
                    'RetryIntervalMs' => <integer>,
                    'SecurityProtocol' => '<string>',
                    'StartingOffsets' => '<string>',
                    'StartingTimestamp' => <integer || string || DateTime>,
                    'SubscribePattern' => '<string>',
                    'TopicName' => '<string>',
                ],
                'Table' => '<string>', // REQUIRED
                'WindowSize' => <integer>,
            ],
            'CatalogKinesisSource' => [
                'DataPreviewOptions' => [
                    'PollingTime' => <integer>,
                    'RecordPollingLimit' => <integer>,
                ],
                'Database' => '<string>', // REQUIRED
                'DetectSchema' => true || false,
                'Name' => '<string>', // REQUIRED
                'StreamingOptions' => [
                    'AddIdleTimeBetweenReads' => true || false,
                    'AddRecordTimestamp' => '<string>',
                    'AvoidEmptyBatches' => true || false,
                    'Classification' => '<string>',
                    'Delimiter' => '<string>',
                    'DescribeShardInterval' => <integer>,
                    'EmitConsumerLagMetrics' => '<string>',
                    'EndpointUrl' => '<string>',
                    'IdleTimeBetweenReadsInMs' => <integer>,
                    'MaxFetchRecordsPerShard' => <integer>,
                    'MaxFetchTimeInMs' => <integer>,
                    'MaxRecordPerRead' => <integer>,
                    'MaxRetryIntervalMs' => <integer>,
                    'NumRetries' => <integer>,
                    'RetryIntervalMs' => <integer>,
                    'RoleArn' => '<string>',
                    'RoleSessionName' => '<string>',
                    'StartingPosition' => 'latest|trim_horizon|earliest|timestamp',
                    'StartingTimestamp' => <integer || string || DateTime>,
                    'StreamArn' => '<string>',
                    'StreamName' => '<string>',
                ],
                'Table' => '<string>', // REQUIRED
                'WindowSize' => <integer>,
            ],
            'CatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'CatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'ConnectorDataSource' => [
                'ConnectionType' => '<string>', // REQUIRED
                'Data' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'ConnectorDataTarget' => [
                'ConnectionType' => '<string>', // REQUIRED
                'Data' => ['<string>', ...], // REQUIRED
                'Inputs' => ['<string>', ...],
                'Name' => '<string>', // REQUIRED
            ],
            'CustomCode' => [
                'ClassName' => '<string>', // REQUIRED
                'Code' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'DirectJDBCSource' => [
                'ConnectionName' => '<string>', // REQUIRED
                'ConnectionType' => 'sqlserver|mysql|oracle|postgresql|redshift', // REQUIRED
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'RedshiftTmpDir' => '<string>',
                'Table' => '<string>', // REQUIRED
            ],
            'DirectKafkaSource' => [
                'DataPreviewOptions' => [
                    'PollingTime' => <integer>,
                    'RecordPollingLimit' => <integer>,
                ],
                'DetectSchema' => true || false,
                'Name' => '<string>', // REQUIRED
                'StreamingOptions' => [
                    'AddRecordTimestamp' => '<string>',
                    'Assign' => '<string>',
                    'BootstrapServers' => '<string>',
                    'Classification' => '<string>',
                    'ConnectionName' => '<string>',
                    'Delimiter' => '<string>',
                    'EmitConsumerLagMetrics' => '<string>',
                    'EndingOffsets' => '<string>',
                    'IncludeHeaders' => true || false,
                    'MaxOffsetsPerTrigger' => <integer>,
                    'MinPartitions' => <integer>,
                    'NumRetries' => <integer>,
                    'PollTimeoutMs' => <integer>,
                    'RetryIntervalMs' => <integer>,
                    'SecurityProtocol' => '<string>',
                    'StartingOffsets' => '<string>',
                    'StartingTimestamp' => <integer || string || DateTime>,
                    'SubscribePattern' => '<string>',
                    'TopicName' => '<string>',
                ],
                'WindowSize' => <integer>,
            ],
            'DirectKinesisSource' => [
                'DataPreviewOptions' => [
                    'PollingTime' => <integer>,
                    'RecordPollingLimit' => <integer>,
                ],
                'DetectSchema' => true || false,
                'Name' => '<string>', // REQUIRED
                'StreamingOptions' => [
                    'AddIdleTimeBetweenReads' => true || false,
                    'AddRecordTimestamp' => '<string>',
                    'AvoidEmptyBatches' => true || false,
                    'Classification' => '<string>',
                    'Delimiter' => '<string>',
                    'DescribeShardInterval' => <integer>,
                    'EmitConsumerLagMetrics' => '<string>',
                    'EndpointUrl' => '<string>',
                    'IdleTimeBetweenReadsInMs' => <integer>,
                    'MaxFetchRecordsPerShard' => <integer>,
                    'MaxFetchTimeInMs' => <integer>,
                    'MaxRecordPerRead' => <integer>,
                    'MaxRetryIntervalMs' => <integer>,
                    'NumRetries' => <integer>,
                    'RetryIntervalMs' => <integer>,
                    'RoleArn' => '<string>',
                    'RoleSessionName' => '<string>',
                    'StartingPosition' => 'latest|trim_horizon|earliest|timestamp',
                    'StartingTimestamp' => <integer || string || DateTime>,
                    'StreamArn' => '<string>',
                    'StreamName' => '<string>',
                ],
                'WindowSize' => <integer>,
            ],
            'DropDuplicates' => [
                'Columns' => [
                    ['<string>', ...],
                    // ...
                ],
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
            ],
            'DropFields' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Paths' => [ // REQUIRED
                    ['<string>', ...],
                    // ...
                ],
            ],
            'DropNullFields' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'NullCheckBoxList' => [
                    'IsEmpty' => true || false,
                    'IsNegOne' => true || false,
                    'IsNullString' => true || false,
                ],
                'NullTextList' => [
                    [
                        'Datatype' => [ // REQUIRED
                            'Id' => '<string>', // REQUIRED
                            'Label' => '<string>', // REQUIRED
                        ],
                        'Value' => '<string>', // REQUIRED
                    ],
                    // ...
                ],
            ],
            'DynamicTransform' => [
                'FunctionName' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Parameters' => [
                    [
                        'IsOptional' => true || false,
                        'ListType' => 'str|int|float|complex|bool|list|null',
                        'Name' => '<string>', // REQUIRED
                        'Type' => 'str|int|float|complex|bool|list|null', // REQUIRED
                        'ValidationMessage' => '<string>',
                        'ValidationRule' => '<string>',
                        'Value' => ['<string>', ...],
                    ],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'TransformName' => '<string>', // REQUIRED
                'Version' => '<string>',
            ],
            'DynamoDBCatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'EvaluateDataQuality' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Output' => 'PrimaryInput|EvaluationResults',
                'PublishingOptions' => [
                    'CloudWatchMetricsEnabled' => true || false,
                    'EvaluationContext' => '<string>',
                    'ResultsPublishingEnabled' => true || false,
                    'ResultsS3Prefix' => '<string>',
                ],
                'Ruleset' => '<string>', // REQUIRED
                'StopJobOnFailureOptions' => [
                    'StopJobOnFailureTiming' => 'Immediate|AfterDataLoad',
                ],
            ],
            'EvaluateDataQualityMultiFrame' => [
                'AdditionalDataSources' => ['<string>', ...],
                'AdditionalOptions' => ['<string>', ...],
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PublishingOptions' => [
                    'CloudWatchMetricsEnabled' => true || false,
                    'EvaluationContext' => '<string>',
                    'ResultsPublishingEnabled' => true || false,
                    'ResultsS3Prefix' => '<string>',
                ],
                'Ruleset' => '<string>', // REQUIRED
                'StopJobOnFailureOptions' => [
                    'StopJobOnFailureTiming' => 'Immediate|AfterDataLoad',
                ],
            ],
            'FillMissingValues' => [
                'FilledPath' => '<string>',
                'ImputedPath' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
            ],
            'Filter' => [
                'Filters' => [ // REQUIRED
                    [
                        'Negated' => true || false,
                        'Operation' => 'EQ|LT|GT|LTE|GTE|REGEX|ISNULL', // REQUIRED
                        'Values' => [ // REQUIRED
                            [
                                'Type' => 'COLUMNEXTRACTED|CONSTANT', // REQUIRED
                                'Value' => ['<string>', ...], // REQUIRED
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Inputs' => ['<string>', ...], // REQUIRED
                'LogicalOperator' => 'AND|OR', // REQUIRED
                'Name' => '<string>', // REQUIRED
            ],
            'GovernedCatalogSource' => [
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                ],
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionPredicate' => '<string>',
                'Table' => '<string>', // REQUIRED
            ],
            'GovernedCatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'SchemaChangePolicy' => [
                    'EnableUpdateCatalog' => true || false,
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'JDBCConnectorSource' => [
                'AdditionalOptions' => [
                    'DataTypeMapping' => ['<string>', ...],
                    'FilterPredicate' => '<string>',
                    'JobBookmarkKeys' => ['<string>', ...],
                    'JobBookmarkKeysSortOrder' => '<string>',
                    'LowerBound' => <integer>,
                    'NumPartitions' => <integer>,
                    'PartitionColumn' => '<string>',
                    'UpperBound' => <integer>,
                ],
                'ConnectionName' => '<string>', // REQUIRED
                'ConnectionTable' => '<string>',
                'ConnectionType' => '<string>', // REQUIRED
                'ConnectorName' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Query' => '<string>',
            ],
            'JDBCConnectorTarget' => [
                'AdditionalOptions' => ['<string>', ...],
                'ConnectionName' => '<string>', // REQUIRED
                'ConnectionTable' => '<string>', // REQUIRED
                'ConnectionType' => '<string>', // REQUIRED
                'ConnectorName' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'Join' => [
                'Columns' => [ // REQUIRED
                    [
                        'From' => '<string>', // REQUIRED
                        'Keys' => [ // REQUIRED
                            ['<string>', ...],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Inputs' => ['<string>', ...], // REQUIRED
                'JoinType' => 'equijoin|left|right|outer|leftsemi|leftanti', // REQUIRED
                'Name' => '<string>', // REQUIRED
            ],
            'Merge' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PrimaryKeys' => [ // REQUIRED
                    ['<string>', ...],
                    // ...
                ],
                'Source' => '<string>', // REQUIRED
            ],
            'MicrosoftSQLServerCatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'MicrosoftSQLServerCatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'MySQLCatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'MySQLCatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'OracleSQLCatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'OracleSQLCatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'PIIDetection' => [
                'EntityTypesToDetect' => ['<string>', ...], // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'MaskValue' => '<string>',
                'Name' => '<string>', // REQUIRED
                'OutputColumnName' => '<string>',
                'PiiType' => 'RowAudit|RowMasking|ColumnAudit|ColumnMasking', // REQUIRED
                'SampleFraction' => <float>,
                'ThresholdFraction' => <float>,
            ],
            'PostgreSQLCatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'PostgreSQLCatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'Recipe' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'RecipeReference' => [
                    'RecipeArn' => '<string>', // REQUIRED
                    'RecipeVersion' => '<string>', // REQUIRED
                ],
                'RecipeSteps' => [
                    [
                        'Action' => [ // REQUIRED
                            'Operation' => '<string>', // REQUIRED
                            'Parameters' => ['<string>', ...],
                        ],
                        'ConditionExpressions' => [
                            [
                                'Condition' => '<string>', // REQUIRED
                                'TargetColumn' => '<string>', // REQUIRED
                                'Value' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'RedshiftSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'RedshiftTmpDir' => '<string>',
                'Table' => '<string>', // REQUIRED
                'TmpDirIAMRole' => '<string>',
            ],
            'RedshiftTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'RedshiftTmpDir' => '<string>',
                'Table' => '<string>', // REQUIRED
                'TmpDirIAMRole' => '<string>',
                'UpsertRedshiftOptions' => [
                    'ConnectionName' => '<string>',
                    'TableLocation' => '<string>',
                    'UpsertKeys' => ['<string>', ...],
                ],
            ],
            'RelationalCatalogSource' => [
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Table' => '<string>', // REQUIRED
            ],
            'RenameField' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'SourcePath' => ['<string>', ...], // REQUIRED
                'TargetPath' => ['<string>', ...], // REQUIRED
            ],
            'S3CatalogDeltaSource' => [
                'AdditionalDeltaOptions' => ['<string>', ...],
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'S3CatalogHudiSource' => [
                'AdditionalHudiOptions' => ['<string>', ...],
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'S3CatalogSource' => [
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                ],
                'Database' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionPredicate' => '<string>',
                'Table' => '<string>', // REQUIRED
            ],
            'S3CatalogTarget' => [
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'SchemaChangePolicy' => [
                    'EnableUpdateCatalog' => true || false,
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'S3CsvSource' => [
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                    'EnableSamplePath' => true || false,
                    'SamplePath' => '<string>',
                ],
                'CompressionType' => 'gzip|bzip2',
                'Escaper' => '<string>',
                'Exclusions' => ['<string>', ...],
                'GroupFiles' => '<string>',
                'GroupSize' => '<string>',
                'MaxBand' => <integer>,
                'MaxFilesInBand' => <integer>,
                'Multiline' => true || false,
                'Name' => '<string>', // REQUIRED
                'OptimizePerformance' => true || false,
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Paths' => ['<string>', ...], // REQUIRED
                'QuoteChar' => 'quote|quillemet|single_quote|disabled', // REQUIRED
                'Recurse' => true || false,
                'Separator' => 'comma|ctrla|pipe|semicolon|tab', // REQUIRED
                'SkipFirst' => true || false,
                'WithHeader' => true || false,
                'WriteHeader' => true || false,
            ],
            'S3DeltaCatalogTarget' => [
                'AdditionalOptions' => ['<string>', ...],
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'SchemaChangePolicy' => [
                    'EnableUpdateCatalog' => true || false,
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'S3DeltaDirectTarget' => [
                'AdditionalOptions' => ['<string>', ...],
                'Compression' => 'uncompressed|snappy', // REQUIRED
                'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'NumberTargetPartitions' => '<string>',
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'SchemaChangePolicy' => [
                    'Database' => '<string>',
                    'EnableUpdateCatalog' => true || false,
                    'Table' => '<string>',
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
            ],
            'S3DeltaSource' => [
                'AdditionalDeltaOptions' => ['<string>', ...],
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                    'EnableSamplePath' => true || false,
                    'SamplePath' => '<string>',
                ],
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Paths' => ['<string>', ...], // REQUIRED
            ],
            'S3DirectTarget' => [
                'Compression' => '<string>',
                'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'NumberTargetPartitions' => '<string>',
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'SchemaChangePolicy' => [
                    'Database' => '<string>',
                    'EnableUpdateCatalog' => true || false,
                    'Table' => '<string>',
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
            ],
            'S3ExcelSource' => [
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                    'EnableSamplePath' => true || false,
                    'SamplePath' => '<string>',
                ],
                'CompressionType' => 'snappy|lzo|gzip|brotli|lz4|uncompressed|none',
                'Exclusions' => ['<string>', ...],
                'GroupFiles' => '<string>',
                'GroupSize' => '<string>',
                'MaxBand' => <integer>,
                'MaxFilesInBand' => <integer>,
                'Name' => '<string>', // REQUIRED
                'NumberRows' => <integer>,
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Paths' => ['<string>', ...], // REQUIRED
                'Recurse' => true || false,
                'SkipFooter' => <integer>,
            ],
            'S3GlueParquetTarget' => [
                'Compression' => 'snappy|lzo|gzip|brotli|lz4|uncompressed|none',
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'NumberTargetPartitions' => '<string>',
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'SchemaChangePolicy' => [
                    'Database' => '<string>',
                    'EnableUpdateCatalog' => true || false,
                    'Table' => '<string>',
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
            ],
            'S3HudiCatalogTarget' => [
                'AdditionalOptions' => ['<string>', ...], // REQUIRED
                'Database' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'SchemaChangePolicy' => [
                    'EnableUpdateCatalog' => true || false,
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
                'Table' => '<string>', // REQUIRED
            ],
            'S3HudiDirectTarget' => [
                'AdditionalOptions' => ['<string>', ...], // REQUIRED
                'Compression' => 'gzip|lzo|uncompressed|snappy', // REQUIRED
                'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'NumberTargetPartitions' => '<string>',
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'SchemaChangePolicy' => [
                    'Database' => '<string>',
                    'EnableUpdateCatalog' => true || false,
                    'Table' => '<string>',
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
            ],
            'S3HudiSource' => [
                'AdditionalHudiOptions' => ['<string>', ...],
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                    'EnableSamplePath' => true || false,
                    'SamplePath' => '<string>',
                ],
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Paths' => ['<string>', ...], // REQUIRED
            ],
            'S3HyperDirectTarget' => [
                'Compression' => 'uncompressed',
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'SchemaChangePolicy' => [
                    'Database' => '<string>',
                    'EnableUpdateCatalog' => true || false,
                    'Table' => '<string>',
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
            ],
            'S3IcebergDirectTarget' => [
                'AdditionalOptions' => ['<string>', ...],
                'Compression' => 'gzip|lzo|uncompressed|snappy', // REQUIRED
                'Format' => 'json|csv|avro|orc|parquet|hudi|delta|iceberg|hyper|xml', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'NumberTargetPartitions' => '<string>',
                'PartitionKeys' => [
                    ['<string>', ...],
                    // ...
                ],
                'Path' => '<string>', // REQUIRED
                'SchemaChangePolicy' => [
                    'Database' => '<string>',
                    'EnableUpdateCatalog' => true || false,
                    'Table' => '<string>',
                    'UpdateBehavior' => 'UPDATE_IN_DATABASE|LOG',
                ],
            ],
            'S3JsonSource' => [
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                    'EnableSamplePath' => true || false,
                    'SamplePath' => '<string>',
                ],
                'CompressionType' => 'gzip|bzip2',
                'Exclusions' => ['<string>', ...],
                'GroupFiles' => '<string>',
                'GroupSize' => '<string>',
                'JsonPath' => '<string>',
                'MaxBand' => <integer>,
                'MaxFilesInBand' => <integer>,
                'Multiline' => true || false,
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Paths' => ['<string>', ...], // REQUIRED
                'Recurse' => true || false,
            ],
            'S3ParquetSource' => [
                'AdditionalOptions' => [
                    'BoundedFiles' => <integer>,
                    'BoundedSize' => <integer>,
                    'EnableSamplePath' => true || false,
                    'SamplePath' => '<string>',
                ],
                'CompressionType' => 'snappy|lzo|gzip|brotli|lz4|uncompressed|none',
                'Exclusions' => ['<string>', ...],
                'GroupFiles' => '<string>',
                'GroupSize' => '<string>',
                'MaxBand' => <integer>,
                'MaxFilesInBand' => <integer>,
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'Paths' => ['<string>', ...], // REQUIRED
                'Recurse' => true || false,
            ],
            'SelectFields' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Paths' => [ // REQUIRED
                    ['<string>', ...],
                    // ...
                ],
            ],
            'SelectFromCollection' => [
                'Index' => <integer>, // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
            ],
            'SnowflakeSource' => [
                'Data' => [ // REQUIRED
                    'Action' => '<string>',
                    'AdditionalOptions' => ['<string>', ...],
                    'AutoPushdown' => true || false,
                    'Connection' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'Database' => '<string>',
                    'IamRole' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'MergeAction' => '<string>',
                    'MergeClause' => '<string>',
                    'MergeWhenMatched' => '<string>',
                    'MergeWhenNotMatched' => '<string>',
                    'PostAction' => '<string>',
                    'PreAction' => '<string>',
                    'SampleQuery' => '<string>',
                    'Schema' => '<string>',
                    'SelectedColumns' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'SourceType' => '<string>',
                    'StagingTable' => '<string>',
                    'Table' => '<string>',
                    'TableSchema' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'TempDir' => '<string>',
                    'Upsert' => true || false,
                ],
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'SnowflakeTarget' => [
                'Data' => [ // REQUIRED
                    'Action' => '<string>',
                    'AdditionalOptions' => ['<string>', ...],
                    'AutoPushdown' => true || false,
                    'Connection' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'Database' => '<string>',
                    'IamRole' => [
                        'Description' => '<string>',
                        'Label' => '<string>',
                        'Value' => '<string>',
                    ],
                    'MergeAction' => '<string>',
                    'MergeClause' => '<string>',
                    'MergeWhenMatched' => '<string>',
                    'MergeWhenNotMatched' => '<string>',
                    'PostAction' => '<string>',
                    'PreAction' => '<string>',
                    'SampleQuery' => '<string>',
                    'Schema' => '<string>',
                    'SelectedColumns' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'SourceType' => '<string>',
                    'StagingTable' => '<string>',
                    'Table' => '<string>',
                    'TableSchema' => [
                        [
                            'Description' => '<string>',
                            'Label' => '<string>',
                            'Value' => '<string>',
                        ],
                        // ...
                    ],
                    'TempDir' => '<string>',
                    'Upsert' => true || false,
                ],
                'Inputs' => ['<string>', ...],
                'Name' => '<string>', // REQUIRED
            ],
            'SparkConnectorSource' => [
                'AdditionalOptions' => ['<string>', ...],
                'ConnectionName' => '<string>', // REQUIRED
                'ConnectionType' => '<string>', // REQUIRED
                'ConnectorName' => '<string>', // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'SparkConnectorTarget' => [
                'AdditionalOptions' => ['<string>', ...],
                'ConnectionName' => '<string>', // REQUIRED
                'ConnectionType' => '<string>', // REQUIRED
                'ConnectorName' => '<string>', // REQUIRED
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
            ],
            'SparkSQL' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'OutputSchemas' => [
                    [
                        'Columns' => [
                            [
                                'Name' => '<string>', // REQUIRED
                                'Type' => '<string>',
                            ],
                            // ...
                        ],
                    ],
                    // ...
                ],
                'SqlAliases' => [ // REQUIRED
                    [
                        'Alias' => '<string>', // REQUIRED
                        'From' => '<string>', // REQUIRED
                    ],
                    // ...
                ],
                'SqlQuery' => '<string>', // REQUIRED
            ],
            'Spigot' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Path' => '<string>', // REQUIRED
                'Prob' => <float>,
                'Topk' => <integer>,
            ],
            'SplitFields' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'Paths' => [ // REQUIRED
                    ['<string>', ...],
                    // ...
                ],
            ],
            'Union' => [
                'Inputs' => ['<string>', ...], // REQUIRED
                'Name' => '<string>', // REQUIRED
                'UnionType' => 'ALL|DISTINCT', // REQUIRED
            ],
        ],
        // ...
    ],
    'Command' => [ // REQUIRED
        'Name' => '<string>',
        'PythonVersion' => '<string>',
        'Runtime' => '<string>',
        'ScriptLocation' => '<string>',
    ],
    'Connections' => [
        'Connections' => ['<string>', ...],
    ],
    'DefaultArguments' => ['<string>', ...],
    'Description' => '<string>',
    'ExecutionClass' => 'FLEX|STANDARD',
    'ExecutionProperty' => [
        'MaxConcurrentRuns' => <integer>,
    ],
    'GlueVersion' => '<string>',
    'JobMode' => 'SCRIPT|VISUAL|NOTEBOOK',
    'JobRunQueuingEnabled' => true || false,
    'LogUri' => '<string>',
    'MaintenanceWindow' => '<string>',
    'MaxCapacity' => <float>,
    'MaxRetries' => <integer>,
    'Name' => '<string>', // REQUIRED
    'NonOverridableArguments' => ['<string>', ...],
    'NotificationProperty' => [
        'NotifyDelayAfter' => <integer>,
    ],
    'NumberOfWorkers' => <integer>,
    'Role' => '<string>', // REQUIRED
    'SecurityConfiguration' => '<string>',
    'SourceControlDetails' => [
        'AuthStrategy' => 'PERSONAL_ACCESS_TOKEN|AWS_SECRETS_MANAGER',
        'AuthToken' => '<string>',
        'Branch' => '<string>',
        'Folder' => '<string>',
        'LastCommitId' => '<string>',
        'Owner' => '<string>',
        'Provider' => 'GITHUB|GITLAB|BITBUCKET|AWS_CODE_COMMIT',
        'Repository' => '<string>',
    ],
    'Tags' => ['<string>', ...],
    'Timeout' => <integer>,
    'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
]);

Parameter Details

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

IdempotentParameterMismatchException:

The same unique identifier was associated with two different records.

AlreadyExistsException:

A resource to be created or added already exists.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

ConcurrentModificationException:

Two processes are trying to modify a resource simultaneously.

CreateMLTransform

$result = $client->createMLTransform([/* ... */]);
$promise = $client->createMLTransformAsync([/* ... */]);

Creates an Glue machine learning transform. This operation creates the transform and all the necessary parameters to train it.

Call this operation as the first step in the process of using a machine learning transform (such as the FindMatches transform) for deduplicating data. You can provide an optional Description, in addition to the parameters that you want to use for your algorithm.

You must also specify certain parameters for the tasks that Glue runs on your behalf as part of learning from your data and creating a high-quality machine learning transform. These parameters include Role, and optionally, AllocatedCapacity, Timeout, and MaxRetries. For more information, see Jobs.

Parameter Syntax

$result = $client->createMLTransform([
    'Description' => '<string>',
    'GlueVersion' => '<string>',
    'InputRecordTables' => [ // REQUIRED
        [
            'AdditionalOptions' => ['<string>', ...],
            'CatalogId' => '<string>',
            'ConnectionName' => '<string>',
            'DatabaseName' => '<string>', // REQUIRED
            'TableName' => '<string>', // REQUIRED
        ],
        // ...
    ],
    'MaxCapacity' => <float>,
    'MaxRetries' => <integer>,
    'Name' => '<string>', // REQUIRED
    'NumberOfWorkers' => <integer>,
    'Parameters' => [ // REQUIRED
        'FindMatchesParameters' => [
            'AccuracyCostTradeoff' => <float>,
            'EnforceProvidedLabels' => true || false,
            'PrecisionRecallTradeoff' => <float>,
            'PrimaryKeyColumnName' => '<string>',
        ],
        'TransformType' => 'FIND_MATCHES', // REQUIRED
    ],
    'Role' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
    'Timeout' => <integer>,
    'TransformEncryption' => [
        'MlUserDataEncryption' => [
            'KmsKeyId' => '<string>',
            'MlUserDataEncryptionMode' => 'DISABLED|SSE-KMS', // REQUIRED
        ],
        'TaskRunSecurityConfigurationName' => '<string>',
    ],
    'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
]);

Parameter Details

Result Syntax

[
    'TransformId' => '<string>',
]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

OperationTimeoutException:

The operation timed out.

InternalServiceException:

An internal service error occurred.

AccessDeniedException:

Access to a resource was denied.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

IdempotentParameterMismatchException:

The same unique identifier was associated with two different records.

CreatePartition

$result = $client->createPartition([/* ... */]);
$promise = $client->createPartitionAsync([/* ... */]);

Creates a new partition.

Parameter Syntax

$result = $client->createPartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionInput' => [ // REQUIRED
        'LastAccessTime' => <integer || string || DateTime>,
        'LastAnalyzedTime' => <integer || string || DateTime>,
        'Parameters' => ['<string>', ...],
        'StorageDescriptor' => [
            'AdditionalLocations' => ['<string>', ...],
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>', // REQUIRED
                    'Parameters' => ['<string>', ...],
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SchemaReference' => [
                'SchemaId' => [
                    'RegistryName' => '<string>',
                    'SchemaArn' => '<string>',
                    'SchemaName' => '<string>',
                ],
                'SchemaVersionId' => '<string>',
                'SchemaVersionNumber' => <integer>,
            ],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>', // REQUIRED
                    'SortOrder' => <integer>, // REQUIRED
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'Values' => ['<string>', ...],
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InternalServiceException:

An internal service error occurred.

EntityNotFoundException:

A specified entity does not exist

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

CreatePartitionIndex

$result = $client->createPartitionIndex([/* ... */]);
$promise = $client->createPartitionIndexAsync([/* ... */]);

Creates a specified partition index in an existing table.

Parameter Syntax

$result = $client->createPartitionIndex([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionIndex' => [ // REQUIRED
        'IndexName' => '<string>', // REQUIRED
        'Keys' => ['<string>', ...], // REQUIRED
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

EntityNotFoundException:

A specified entity does not exist

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

CreateRegistry

$result = $client->createRegistry([/* ... */]);
$promise = $client->createRegistryAsync([/* ... */]);

Creates a new registry which may be used to hold a collection of schemas.

Parameter Syntax

$result = $client->createRegistry([
    'Description' => '<string>',
    'RegistryName' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[
    'Description' => '<string>',
    'RegistryArn' => '<string>',
    'RegistryName' => '<string>',
    'Tags' => ['<string>', ...],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AccessDeniedException:

Access to a resource was denied.

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

ConcurrentModificationException:

Two processes are trying to modify a resource simultaneously.

InternalServiceException:

An internal service error occurred.

CreateSchema

$result = $client->createSchema([/* ... */]);
$promise = $client->createSchemaAsync([/* ... */]);

Creates a new schema set and registers the schema definition. Returns an error if the schema set already exists without actually registering the version.

When the schema set is created, a version checkpoint will be set to the first version. Compatibility mode "DISABLED" restricts any additional schema versions from being added after the first schema version. For all other compatibility modes, validation of compatibility settings will be applied only from the second version onwards when the RegisterSchemaVersion API is used.

When this API is called without a RegistryId, this will create an entry for a "default-registry" in the registry database tables, if it is not already present.

Parameter Syntax

$result = $client->createSchema([
    'Compatibility' => 'NONE|DISABLED|BACKWARD|BACKWARD_ALL|FORWARD|FORWARD_ALL|FULL|FULL_ALL',
    'DataFormat' => 'AVRO|JSON|PROTOBUF', // REQUIRED
    'Description' => '<string>',
    'RegistryId' => [
        'RegistryArn' => '<string>',
        'RegistryName' => '<string>',
    ],
    'SchemaDefinition' => '<string>',
    'SchemaName' => '<string>', // REQUIRED
    'Tags' => ['<string>', ...],
]);

Parameter Details

Result Syntax

[
    'Compatibility' => 'NONE|DISABLED|BACKWARD|BACKWARD_ALL|FORWARD|FORWARD_ALL|FULL|FULL_ALL',
    'DataFormat' => 'AVRO|JSON|PROTOBUF',
    'Description' => '<string>',
    'LatestSchemaVersion' => <integer>,
    'NextSchemaVersion' => <integer>,
    'RegistryArn' => '<string>',
    'RegistryName' => '<string>',
    'SchemaArn' => '<string>',
    'SchemaCheckpoint' => <integer>,
    'SchemaName' => '<string>',
    'SchemaStatus' => 'AVAILABLE|PENDING|DELETING',
    'SchemaVersionId' => '<string>',
    'SchemaVersionStatus' => 'AVAILABLE|PENDING|FAILURE|DELETING',
    'Tags' => ['<string>', ...],
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

AccessDeniedException:

Access to a resource was denied.

EntityNotFoundException:

A specified entity does not exist

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

ConcurrentModificationException:

Two processes are trying to modify a resource simultaneously.

InternalServiceException:

An internal service error occurred.

CreateScript

$result = $client->createScript([/* ... */]);
$promise = $client->createScriptAsync([/* ... */]);

Transforms a directed acyclic graph (DAG) into code.

Parameter Syntax

$result = $client->createScript([
    'DagEdges' => [
        [
            'Source' => '<string>', // REQUIRED
            'Target' => '<string>', // REQUIRED
            'TargetParameter' => '<string>',
        ],
        // ...
    ],
    'DagNodes' => [
        [
            'Args' => [ // REQUIRED
                [
                    'Name' => '<string>', // REQUIRED
                    'Param' => true || false,
                    'Value' => '<string>', // REQUIRED
                ],
                // ...
            ],
            'Id' => '<string>', // REQUIRED
            'LineNumber' => <integer>,
            'NodeType' => '<string>', // REQUIRED
        ],
        // ...
    ],
    'Language' => 'PYTHON|SCALA',
]);

Parameter Details

Result Syntax

[
    'PythonScript' => '<string>',
    'ScalaCode' => '<string>',
]

Result Details

Errors

InvalidInputException:

The input provided was not valid.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

CreateSecurityConfiguration

$result = $client->createSecurityConfiguration([/* ... */]);
$promise = $client->createSecurityConfigurationAsync([/* ... */]);

Creates a new security configuration. A security configuration is a set of security properties that can be used by Glue. You can use a security configuration to encrypt data at rest. For information about using security configurations in Glue, see Encrypting Data Written by Crawlers, Jobs, and Development Endpoints.

Parameter Syntax

$result = $client->createSecurityConfiguration([
    'EncryptionConfiguration' => [ // REQUIRED
        'CloudWatchEncryption' => [
            'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
            'KmsKeyArn' => '<string>',
        ],
        'DataQualityEncryption' => [
            'DataQualityEncryptionMode' => 'DISABLED|SSE-KMS',
            'KmsKeyArn' => '<string>',
        ],
        'JobBookmarksEncryption' => [
            'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
            'KmsKeyArn' => '<string>',
        ],
        'S3Encryption' => [
            [
                'KmsKeyArn' => '<string>',
                'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
            ],
            // ...
        ],
    ],
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Result Syntax

[
    'CreatedTimestamp' => <DateTime>,
    'Name' => '<string>',
]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateSession

$result = $client->createSession([/* ... */]);
$promise = $client->createSessionAsync([/* ... */]);

Creates a new session.

Parameter Syntax

$result = $client->createSession([
    'Command' => [ // REQUIRED
        'Name' => '<string>',
        'PythonVersion' => '<string>',
    ],
    'Connections' => [
        'Connections' => ['<string>', ...],
    ],
    'DefaultArguments' => ['<string>', ...],
    'Description' => '<string>',
    'GlueVersion' => '<string>',
    'Id' => '<string>', // REQUIRED
    'IdleTimeout' => <integer>,
    'MaxCapacity' => <float>,
    'NumberOfWorkers' => <integer>,
    'RequestOrigin' => '<string>',
    'Role' => '<string>', // REQUIRED
    'SecurityConfiguration' => '<string>',
    'Tags' => ['<string>', ...],
    'Timeout' => <integer>,
    'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
]);

Parameter Details

Result Syntax

[
    'Session' => [
        'Command' => [
            'Name' => '<string>',
            'PythonVersion' => '<string>',
        ],
        'CompletedOn' => <DateTime>,
        'Connections' => [
            'Connections' => ['<string>', ...],
        ],
        'CreatedOn' => <DateTime>,
        'DPUSeconds' => <float>,
        'DefaultArguments' => ['<string>', ...],
        'Description' => '<string>',
        'ErrorMessage' => '<string>',
        'ExecutionTime' => <float>,
        'GlueVersion' => '<string>',
        'Id' => '<string>',
        'IdleTimeout' => <integer>,
        'MaxCapacity' => <float>,
        'NumberOfWorkers' => <integer>,
        'ProfileName' => '<string>',
        'Progress' => <float>,
        'Role' => '<string>',
        'SecurityConfiguration' => '<string>',
        'Status' => 'PROVISIONING|READY|FAILED|TIMEOUT|STOPPING|STOPPED',
        'WorkerType' => 'Standard|G.1X|G.2X|G.025X|G.4X|G.8X|Z.2X',
    ],
]

Result Details

Errors

AccessDeniedException:

Access to a resource was denied.

IdempotentParameterMismatchException:

The same unique identifier was associated with two different records.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

InvalidInputException:

The input provided was not valid.

ValidationException:

A value could not be validated.

AlreadyExistsException:

A resource to be created or added already exists.

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

CreateTable

$result = $client->createTable([/* ... */]);
$promise = $client->createTableAsync([/* ... */]);

Creates a new table definition in the Data Catalog.

Parameter Syntax

$result = $client->createTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'OpenTableFormatInput' => [
        'IcebergInput' => [
            'MetadataOperation' => 'CREATE', // REQUIRED
            'Version' => '<string>',
        ],
    ],
    'PartitionIndexes' => [
        [
            'IndexName' => '<string>', // REQUIRED
            'Keys' => ['<string>', ...], // REQUIRED
        ],
        // ...
    ],
    'TableInput' => [ // REQUIRED
        'Description' => '<string>',
        'LastAccessTime' => <integer || string || DateTime>,
        'LastAnalyzedTime' => <integer || string || DateTime>,
        'Name' => '<string>', // REQUIRED
        'Owner' => '<string>',
        'Parameters' => ['<string>', ...],
        'PartitionKeys' => [
            [
                'Comment' => '<string>',
                'Name' => '<string>', // REQUIRED
                'Parameters' => ['<string>', ...],
                'Type' => '<string>',
            ],
            // ...
        ],
        'Retention' => <integer>,
        'StorageDescriptor' => [
            'AdditionalLocations' => ['<string>', ...],
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>', // REQUIRED
                    'Parameters' => ['<string>', ...],
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SchemaReference' => [
                'SchemaId' => [
                    'RegistryName' => '<string>',
                    'SchemaArn' => '<string>',
                    'SchemaName' => '<string>',
                ],
                'SchemaVersionId' => '<string>',
                'SchemaVersionNumber' => <integer>,
            ],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>', // REQUIRED
                    'SortOrder' => <integer>, // REQUIRED
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'TableType' => '<string>',
        'TargetTable' => [
            'CatalogId' => '<string>',
            'DatabaseName' => '<string>',
            'Name' => '<string>',
            'Region' => '<string>',
        ],
        'ViewDefinition' => [
            'Definer' => '<string>',
            'IsProtected' => true || false,
            'Representations' => [
                [
                    'Dialect' => 'REDSHIFT|ATHENA|SPARK',
                    'DialectVersion' => '<string>',
                    'ValidationConnection' => '<string>',
                    'ViewExpandedText' => '<string>',
                    'ViewOriginalText' => '<string>',
                ],
                // ...
            ],
            'SubObjects' => ['<string>', ...],
        ],
        'ViewExpandedText' => '<string>',
        'ViewOriginalText' => '<string>',
    ],
    'TransactionId' => '<string>',
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

AlreadyExistsException:

A resource to be created or added already exists.

InvalidInputException:

The input provided was not valid.

EntityNotFoundException:

A specified entity does not exist

ResourceNumberLimitExceededException:

A resource numerical limit was exceeded.

InternalServiceException:

An internal service error occurred.

OperationTimeoutException:

The operation timed out.

GlueEncryptionException:

An encryption operation failed.

ConcurrentModificationException:

Two processes are trying to modify a resource simultaneously.

ResourceNotReadyException:

A resource was not ready for a transaction.

FederationSourceException:

A federation source failed.

FederationSourceRetryableException:

A federation source failed, but the operation may be retried.

CreateTableOptimizer

$result = $client->createTableOptimizer([/* ... */]);
$promise = $client->createTableOptimizerAsync([/* ... */]);

Creates a new table optimizer for a specific function.

Parameter Syntax

$result = $client->createTableOptimizer([
    'CatalogId' => '<string>', // REQUIRED
    'DatabaseName' => '<string>', // REQUIRED
    'TableName' => '<string>', // REQUIRED
    'TableOptimizerConfiguration' => [ // REQUIRED
        'enabled' => true || false,
        'orphanFileDeletionConfiguration' => [
            'icebergConfiguration' => [
                'location' => '<string>',
                'orphanFileRetentionPeriodInDays' => <integer>,
            ],
        ],
        'retentionConfiguration' => [
            'icebergConfiguration' => [
                'cleanExpiredFiles' => true || false,
                'numberOfSnapshotsToRetain' => <integer>,
                'snapshotRetentionPeriodInDays' => <integer>,
            ],
        ],
        'roleArn' => '<string>',
        'vpcConfiguration' => [
            'glueConnectionName' => '<string>',
        ],
    ],
    'Type' => 'compaction|retention|orphan_file_deletion', // REQUIRED
]);

Parameter Details

Result Syntax

[]

Result Details

Errors

EntityNotFoundException:

A specified entity does not exist

ValidationException:

A value could not be validated.

InvalidInputException:

The input provided was not valid.

AccessDeniedException:

Access to a resource was denied.

AlreadyExistsException:

A resource to be created or added already exists.

InternalServiceException:

An internal service error occurred.

ThrottlingException:

The throttling threshhold was exceeded.