AWS SDK for PHP 3.x

Service ID: glue

Version: 2017-03-31

This page describes the parameters and results for the operations of the AWS Glue (2017-03-31), and shows how to use the Aws\Glue\GlueClient object to call the described operations. This documentation is specific to the 2017-03-31 API version of the service.

Operation Summary

Each of the following operations can be created from a client using $client->getCommand('CommandName'), where "CommandName" is the name of one of the following operations. Note: a command is a value that encapsulates an operation and the parameters used to create an HTTP request.

You can also create and send a command immediately using the magic methods available on a client object: $client->commandName(/* parameters */). You can send the command asynchronously (returning a promise) by appending the word "Async" to the operation name: $client->commandNameAsync(/* parameters */).

BatchCreatePartition ( array $params = [] )
Creates one or more partitions in a batch operation.
BatchDeleteConnection ( array $params = [] )
Deletes a list of connection definitions from the Data Catalog.
BatchDeletePartition ( array $params = [] )
Deletes one or more partitions in a batch operation.
BatchDeleteTable ( array $params = [] )
Deletes multiple tables at once.
BatchDeleteTableVersion ( array $params = [] )
Deletes a specified batch of versions of a table.
BatchGetCrawlers ( array $params = [] )
Returns a list of resource metadata for a given list of crawler names.
BatchGetDevEndpoints ( array $params = [] )
Returns a list of resource metadata for a given list of DevEndpoint names.
BatchGetJobs ( array $params = [] )
Returns a list of resource metadata for a given list of job names.
BatchGetPartition ( array $params = [] )
Retrieves partitions in a batch request.
BatchGetTriggers ( array $params = [] )
Returns a list of resource metadata for a given list of trigger names.
BatchStopJobRun ( array $params = [] )
Stops one or more job runs for a specified job definition.
CreateClassifier ( array $params = [] )
Creates a classifier in the user's account.
CreateConnection ( array $params = [] )
Creates a connection definition in the Data Catalog.
CreateCrawler ( array $params = [] )
Creates a new crawler with specified targets, role, configuration, and optional schedule.
CreateDatabase ( array $params = [] )
Creates a new database in a Data Catalog.
CreateDevEndpoint ( array $params = [] )
Creates a new DevEndpoint.
CreateJob ( array $params = [] )
Creates a new job definition.
CreatePartition ( array $params = [] )
Creates a new partition.
CreateScript ( array $params = [] )
Transforms a directed acyclic graph (DAG) into code.
CreateSecurityConfiguration ( array $params = [] )
Creates a new security configuration.
CreateTable ( array $params = [] )
Creates a new table definition in the Data Catalog.
CreateTrigger ( array $params = [] )
Creates a new trigger.
CreateUserDefinedFunction ( array $params = [] )
Creates a new function definition in the Data Catalog.
DeleteClassifier ( array $params = [] )
Removes a classifier from the Data Catalog.
DeleteConnection ( array $params = [] )
Deletes a connection from the Data Catalog.
DeleteCrawler ( array $params = [] )
Removes a specified crawler from the Data Catalog, unless the crawler state is RUNNING.
DeleteDatabase ( array $params = [] )
Removes a specified Database from a Data Catalog.
DeleteDevEndpoint ( array $params = [] )
Deletes a specified DevEndpoint.
DeleteJob ( array $params = [] )
Deletes a specified job definition.
DeletePartition ( array $params = [] )
Deletes a specified partition.
DeleteResourcePolicy ( array $params = [] )
Deletes a specified policy.
DeleteSecurityConfiguration ( array $params = [] )
Deletes a specified security configuration.
DeleteTable ( array $params = [] )
Removes a table definition from the Data Catalog.
DeleteTableVersion ( array $params = [] )
Deletes a specified version of a table.
DeleteTrigger ( array $params = [] )
Deletes a specified trigger.
DeleteUserDefinedFunction ( array $params = [] )
Deletes an existing function definition from the Data Catalog.
GetCatalogImportStatus ( array $params = [] )
Retrieves the status of a migration operation.
GetClassifier ( array $params = [] )
Retrieve a classifier by name.
GetClassifiers ( array $params = [] )
Lists all classifier objects in the Data Catalog.
GetConnection ( array $params = [] )
Retrieves a connection definition from the Data Catalog.
GetConnections ( array $params = [] )
Retrieves a list of connection definitions from the Data Catalog.
GetCrawler ( array $params = [] )
Retrieves metadata for a specified crawler.
GetCrawlerMetrics ( array $params = [] )
Retrieves metrics about specified crawlers.
GetCrawlers ( array $params = [] )
Retrieves metadata for all crawlers defined in the customer account.
GetDataCatalogEncryptionSettings ( array $params = [] )
Retrieves the security configuration for a specified catalog.
GetDatabase ( array $params = [] )
Retrieves the definition of a specified database.
GetDatabases ( array $params = [] )
Retrieves all Databases defined in a given Data Catalog.
GetDataflowGraph ( array $params = [] )
Transforms a Python script into a directed acyclic graph (DAG).
GetDevEndpoint ( array $params = [] )
Retrieves information about a specified DevEndpoint.
GetDevEndpoints ( array $params = [] )
Retrieves all the DevEndpoints in this AWS account.
GetJob ( array $params = [] )
Retrieves an existing job definition.
GetJobRun ( array $params = [] )
Retrieves the metadata for a given job run.
GetJobRuns ( array $params = [] )
Retrieves metadata for all runs of a given job definition.
GetJobs ( array $params = [] )
Retrieves all current job definitions.
GetMapping ( array $params = [] )
Creates mappings.
GetPartition ( array $params = [] )
Retrieves information about a specified partition.
GetPartitions ( array $params = [] )
Retrieves information about the partitions in a table.
GetPlan ( array $params = [] )
Gets code to perform a specified mapping.
GetResourcePolicy ( array $params = [] )
Retrieves a specified resource policy.
GetSecurityConfiguration ( array $params = [] )
Retrieves a specified security configuration.
GetSecurityConfigurations ( array $params = [] )
Retrieves a list of all security configurations.
GetTable ( array $params = [] )
Retrieves the Table definition in a Data Catalog for a specified table.
GetTableVersion ( array $params = [] )
Retrieves a specified version of a table.
GetTableVersions ( array $params = [] )
Retrieves a list of strings that identify available versions of a specified table.
GetTables ( array $params = [] )
Retrieves the definitions of some or all of the tables in a given Database.
GetTags ( array $params = [] )
Retrieves a list of tags associated with a resource.
GetTrigger ( array $params = [] )
Retrieves the definition of a trigger.
GetTriggers ( array $params = [] )
Gets all the triggers associated with a job.
GetUserDefinedFunction ( array $params = [] )
Retrieves a specified function definition from the Data Catalog.
GetUserDefinedFunctions ( array $params = [] )
Retrieves a multiple function definitions from the Data Catalog.
ImportCatalogToGlue ( array $params = [] )
Imports an existing Athena Data Catalog to AWS Glue
ListCrawlers ( array $params = [] )
Retrieves the names of all crawler resources in this AWS account, or the resources with the specified tag.
ListDevEndpoints ( array $params = [] )
Retrieves the names of all DevEndpoint resources in this AWS account, or the resources with the specified tag.
ListJobs ( array $params = [] )
Retrieves the names of all job resources in this AWS account, or the resources with the specified tag.
ListTriggers ( array $params = [] )
Retrieves the names of all trigger resources in this AWS account, or the resources with the specified tag.
PutDataCatalogEncryptionSettings ( array $params = [] )
Sets the security configuration for a specified catalog.
PutResourcePolicy ( array $params = [] )
Sets the Data Catalog resource policy for access control.
ResetJobBookmark ( array $params = [] )
Resets a bookmark entry.
StartCrawler ( array $params = [] )
Starts a crawl using the specified crawler, regardless of what is scheduled.
StartCrawlerSchedule ( array $params = [] )
Changes the schedule state of the specified crawler to SCHEDULED, unless the crawler is already running or the schedule state is already SCHEDULED.
StartJobRun ( array $params = [] )
Starts a job run using a job definition.
StartTrigger ( array $params = [] )
Starts an existing trigger.
StopCrawler ( array $params = [] )
If the specified crawler is running, stops the crawl.
StopCrawlerSchedule ( array $params = [] )
Sets the schedule state of the specified crawler to NOT_SCHEDULED, but does not stop the crawler if it is already running.
StopTrigger ( array $params = [] )
Stops a specified trigger.
TagResource ( array $params = [] )
Adds tags to a resource.
UntagResource ( array $params = [] )
Removes tags from a resource.
UpdateClassifier ( array $params = [] )
Modifies an existing classifier (a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field is present).
UpdateConnection ( array $params = [] )
Updates a connection definition in the Data Catalog.
UpdateCrawler ( array $params = [] )
Updates a crawler.
UpdateCrawlerSchedule ( array $params = [] )
Updates the schedule of a crawler using a cron expression.
UpdateDatabase ( array $params = [] )
Updates an existing database definition in a Data Catalog.
UpdateDevEndpoint ( array $params = [] )
Updates a specified DevEndpoint.
UpdateJob ( array $params = [] )
Updates an existing job definition.
UpdatePartition ( array $params = [] )
Updates a partition.
UpdateTable ( array $params = [] )
Updates a metadata table in the Data Catalog.
UpdateTrigger ( array $params = [] )
Updates a trigger definition.
UpdateUserDefinedFunction ( array $params = [] )
Updates an existing function definition in the Data Catalog.

Paginators

Paginators handle automatically iterating over paginated API results. Paginators are associated with specific API operations, and they accept the parameters that the corresponding API operation accepts. You can get a paginator from a client class using getPaginator($paginatorName, $operationParameters). This client supports the following paginators:

GetClassifiers
GetConnections
GetCrawlerMetrics
GetCrawlers
GetDatabases
GetDevEndpoints
GetJobRuns
GetJobs
GetPartitions
GetSecurityConfigurations
GetTableVersions
GetTables
GetTriggers
GetUserDefinedFunctions
ListCrawlers
ListDevEndpoints
ListJobs
ListTriggers

Operations

BatchCreatePartition

$result = $client->batchCreatePartition([/* ... */]);
$promise = $client->batchCreatePartitionAsync([/* ... */]);

Creates one or more partitions in a batch operation.

Parameter Syntax

$result = $client->batchCreatePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionInputList' => [ // REQUIRED
        [
            'LastAccessTime' => <integer || string || DateTime>,
            'LastAnalyzedTime' => <integer || string || DateTime>,
            'Parameters' => ['<string>', ...],
            'StorageDescriptor' => [
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>', // REQUIRED
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>', // REQUIRED
                        'SortOrder' => <integer>, // REQUIRED
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the catalog in which the partion is to be created. Currently, this should be the AWS account ID.

DatabaseName

Required: Yes
Type: string

The name of the metadata database in which the partition is to be created.

PartitionInputList

Required: Yes
Type: Array of PartitionInput structures

A list of PartitionInput structures that define the partitions to be created.

TableName

Required: Yes
Type: string

The name of the metadata table in which the partition is to be created.

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'PartitionValues' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Members

Errors

Type: Array of PartitionError structures

Errors encountered when trying to create the requested partitions.

Errors

InvalidInputException:
The input provided was not valid.
AlreadyExistsException:
A resource to be created or added already exists.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
InternalServiceException:
An internal service error occurred.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

BatchDeleteConnection

$result = $client->batchDeleteConnection([/* ... */]);
$promise = $client->batchDeleteConnectionAsync([/* ... */]);

Deletes a list of connection definitions from the Data Catalog.

Parameter Syntax

$result = $client->batchDeleteConnection([
    'CatalogId' => '<string>',
    'ConnectionNameList' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the connections reside. If none is provided, the AWS account ID is used by default.

ConnectionNameList

Required: Yes
Type: Array of strings

A list of names of the connections to delete.

Result Syntax

[
    'Errors' => [
        '<NameString>' => [
            'ErrorCode' => '<string>',
            'ErrorMessage' => '<string>',
        ],
        // ...
    ],
    'Succeeded' => ['<string>', ...],
]

Result Details

Members

Errors

Type: Associative array of custom strings keys (NameString) to ErrorDetail structures

A map of the names of connections that were not successfully deleted to error details.

Succeeded

Type: Array of strings

A list of names of the connection definitions that were successfully deleted.

Errors

InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

BatchDeletePartition

$result = $client->batchDeletePartition([/* ... */]);
$promise = $client->batchDeletePartitionAsync([/* ... */]);

Deletes one or more partitions in a batch operation.

Parameter Syntax

$result = $client->batchDeletePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionsToDelete' => [ // REQUIRED
        [
            'Values' => ['<string>', ...], // REQUIRED
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the partition to be deleted resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database in which the table in question resides.

PartitionsToDelete

Required: Yes
Type: Array of PartitionValueList structures

A list of PartitionInput structures that define the partitions to be deleted.

TableName

Required: Yes
Type: string

The name of the table where the partitions to be deleted is located.

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'PartitionValues' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Members

Errors

Type: Array of PartitionError structures

Errors encountered when trying to delete the requested partitions.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

BatchDeleteTable

$result = $client->batchDeleteTable([/* ... */]);
$promise = $client->batchDeleteTableAsync([/* ... */]);

Deletes multiple tables at once.

After completing this operation, you will no longer have access to the table versions and partitions that belong to the deleted table. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.

To ensure immediate deletion of all related resources, before calling BatchDeleteTable, use DeleteTableVersion or BatchDeleteTableVersion, and DeletePartition or BatchDeletePartition, to delete any resources that belong to the table.

Parameter Syntax

$result = $client->batchDeleteTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TablesToDelete' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the tables to delete reside. For Hive compatibility, this name is entirely lowercase.

TablesToDelete

Required: Yes
Type: Array of strings

A list of the table to delete.

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'TableName' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

Errors

Type: Array of TableError structures

A list of errors encountered in attempting to delete the specified tables.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

BatchDeleteTableVersion

$result = $client->batchDeleteTableVersion([/* ... */]);
$promise = $client->batchDeleteTableVersionAsync([/* ... */]);

Deletes a specified batch of versions of a table.

Parameter Syntax

$result = $client->batchDeleteTableVersion([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TableName' => '<string>', // REQUIRED
    'VersionIds' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.

TableName

Required: Yes
Type: string

The name of the table. For Hive compatibility, this name is entirely lowercase.

VersionIds

Required: Yes
Type: Array of strings

A list of the IDs of versions to be deleted. A VersionId is a string representation of an integer. Each version is incremented by 1.

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'TableName' => '<string>',
            'VersionId' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

Errors

Type: Array of TableVersionError structures

A list of errors encountered while trying to delete the specified table versions.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

BatchGetCrawlers

$result = $client->batchGetCrawlers([/* ... */]);
$promise = $client->batchGetCrawlersAsync([/* ... */]);

Returns a list of resource metadata for a given list of crawler names. After calling the ListCrawlers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetCrawlers([
    'CrawlerNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

CrawlerNames

Required: Yes
Type: Array of strings

A list of crawler names, which may be the names returned from the ListCrawlers operation.

Result Syntax

[
    'Crawlers' => [
        [
            'Classifiers' => ['<string>', ...],
            'Configuration' => '<string>',
            'CrawlElapsedTime' => <integer>,
            'CrawlerSecurityConfiguration' => '<string>',
            'CreationTime' => <DateTime>,
            'DatabaseName' => '<string>',
            'Description' => '<string>',
            'LastCrawl' => [
                'ErrorMessage' => '<string>',
                'LogGroup' => '<string>',
                'LogStream' => '<string>',
                'MessagePrefix' => '<string>',
                'StartTime' => <DateTime>,
                'Status' => 'SUCCEEDED|CANCELLED|FAILED',
            ],
            'LastUpdated' => <DateTime>,
            'Name' => '<string>',
            'Role' => '<string>',
            'Schedule' => [
                'ScheduleExpression' => '<string>',
                'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
            ],
            'SchemaChangePolicy' => [
                'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
                'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
            ],
            'State' => 'READY|RUNNING|STOPPING',
            'TablePrefix' => '<string>',
            'Targets' => [
                'DynamoDBTargets' => [
                    [
                        'Path' => '<string>',
                    ],
                    // ...
                ],
                'JdbcTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'Exclusions' => ['<string>', ...],
                        'Path' => '<string>',
                    ],
                    // ...
                ],
                'S3Targets' => [
                    [
                        'Exclusions' => ['<string>', ...],
                        'Path' => '<string>',
                    ],
                    // ...
                ],
            ],
            'Version' => <integer>,
        ],
        // ...
    ],
    'CrawlersNotFound' => ['<string>', ...],
]

Result Details

Members

Crawlers

Type: Array of Crawler structures

A list of crawler definitions.

CrawlersNotFound

Type: Array of strings

A list of names of crawlers not found.

Errors

InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.

BatchGetDevEndpoints

$result = $client->batchGetDevEndpoints([/* ... */]);
$promise = $client->batchGetDevEndpointsAsync([/* ... */]);

Returns a list of resource metadata for a given list of DevEndpoint names. After calling the ListDevEndpoints operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetDevEndpoints([
    'DevEndpointNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

DevEndpointNames

Required: Yes
Type: Array of strings

The list of DevEndpoint names, which may be the names returned from the ListDevEndpoint operation.

Result Syntax

[
    'DevEndpoints' => [
        [
            'Arguments' => ['<string>', ...],
            'AvailabilityZone' => '<string>',
            'CreatedTimestamp' => <DateTime>,
            'EndpointName' => '<string>',
            'ExtraJarsS3Path' => '<string>',
            'ExtraPythonLibsS3Path' => '<string>',
            'FailureReason' => '<string>',
            'LastModifiedTimestamp' => <DateTime>,
            'LastUpdateStatus' => '<string>',
            'NumberOfNodes' => <integer>,
            'PrivateAddress' => '<string>',
            'PublicAddress' => '<string>',
            'PublicKey' => '<string>',
            'PublicKeys' => ['<string>', ...],
            'RoleArn' => '<string>',
            'SecurityConfiguration' => '<string>',
            'SecurityGroupIds' => ['<string>', ...],
            'Status' => '<string>',
            'SubnetId' => '<string>',
            'VpcId' => '<string>',
            'YarnEndpointAddress' => '<string>',
            'ZeppelinRemoteSparkInterpreterPort' => <integer>,
        ],
        // ...
    ],
    'DevEndpointsNotFound' => ['<string>', ...],
]

Result Details

Members

DevEndpoints

Type: Array of DevEndpoint structures

A list of DevEndpoint definitions.

DevEndpointsNotFound

Type: Array of strings

A list of DevEndpoints not found.

Errors

AccessDeniedException:
Access to a resource was denied.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

BatchGetJobs

$result = $client->batchGetJobs([/* ... */]);
$promise = $client->batchGetJobsAsync([/* ... */]);

Returns a list of resource metadata for a given list of job names. After calling the ListJobs operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetJobs([
    'JobNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

JobNames

Required: Yes
Type: Array of strings

A list of job names, which may be the names returned from the ListJobs operation.

Result Syntax

[
    'Jobs' => [
        [
            'AllocatedCapacity' => <integer>,
            'Command' => [
                'Name' => '<string>',
                'ScriptLocation' => '<string>',
            ],
            'Connections' => [
                'Connections' => ['<string>', ...],
            ],
            'CreatedOn' => <DateTime>,
            'DefaultArguments' => ['<string>', ...],
            'Description' => '<string>',
            'ExecutionProperty' => [
                'MaxConcurrentRuns' => <integer>,
            ],
            'LastModifiedOn' => <DateTime>,
            'LogUri' => '<string>',
            'MaxCapacity' => <float>,
            'MaxRetries' => <integer>,
            'Name' => '<string>',
            'NotificationProperty' => [
                'NotifyDelayAfter' => <integer>,
            ],
            'NumberOfWorkers' => <integer>,
            'Role' => '<string>',
            'SecurityConfiguration' => '<string>',
            'Timeout' => <integer>,
            'WorkerType' => 'Standard|G.1X|G.2X',
        ],
        // ...
    ],
    'JobsNotFound' => ['<string>', ...],
]

Result Details

Members

Jobs

Type: Array of Job structures

A list of job definitions.

JobsNotFound

Type: Array of strings

A list of names of jobs not found.

Errors

InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

BatchGetPartition

$result = $client->batchGetPartition([/* ... */]);
$promise = $client->batchGetPartitionAsync([/* ... */]);

Retrieves partitions in a batch request.

Parameter Syntax

$result = $client->batchGetPartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionsToGet' => [ // REQUIRED
        [
            'Values' => ['<string>', ...], // REQUIRED
        ],
        // ...
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the partitions reside.

PartitionsToGet

Required: Yes
Type: Array of PartitionValueList structures

A list of partition values identifying the partitions to retrieve.

TableName

Required: Yes
Type: string

The name of the partitions' table.

Result Syntax

[
    'Partitions' => [
        [
            'CreationTime' => <DateTime>,
            'DatabaseName' => '<string>',
            'LastAccessTime' => <DateTime>,
            'LastAnalyzedTime' => <DateTime>,
            'Parameters' => ['<string>', ...],
            'StorageDescriptor' => [
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>',
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>',
                        'SortOrder' => <integer>,
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'TableName' => '<string>',
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
    'UnprocessedKeys' => [
        [
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Members

Partitions

Type: Array of Partition structures

A list of the requested partitions.

UnprocessedKeys

Type: Array of PartitionValueList structures

A list of the partition values in the request for which partions were not returned.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
InternalServiceException:
An internal service error occurred.
GlueEncryptionException:
An encryption operation failed.

BatchGetTriggers

$result = $client->batchGetTriggers([/* ... */]);
$promise = $client->batchGetTriggersAsync([/* ... */]);

Returns a list of resource metadata for a given list of trigger names. After calling the ListTriggers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Parameter Syntax

$result = $client->batchGetTriggers([
    'TriggerNames' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

TriggerNames

Required: Yes
Type: Array of strings

A list of trigger names, which may be the names returned from the ListTriggers operation.

Result Syntax

[
    'Triggers' => [
        [
            'Actions' => [
                [
                    'Arguments' => ['<string>', ...],
                    'JobName' => '<string>',
                    'NotificationProperty' => [
                        'NotifyDelayAfter' => <integer>,
                    ],
                    'SecurityConfiguration' => '<string>',
                    'Timeout' => <integer>,
                ],
                // ...
            ],
            'Description' => '<string>',
            'Id' => '<string>',
            'Name' => '<string>',
            'Predicate' => [
                'Conditions' => [
                    [
                        'JobName' => '<string>',
                        'LogicalOperator' => 'EQUALS',
                        'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
                    ],
                    // ...
                ],
                'Logical' => 'AND|ANY',
            ],
            'Schedule' => '<string>',
            'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
            'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
        ],
        // ...
    ],
    'TriggersNotFound' => ['<string>', ...],
]

Result Details

Members

Triggers

Type: Array of Trigger structures

A list of trigger definitions.

TriggersNotFound

Type: Array of strings

A list of names of triggers not found.

Errors

InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

BatchStopJobRun

$result = $client->batchStopJobRun([/* ... */]);
$promise = $client->batchStopJobRunAsync([/* ... */]);

Stops one or more job runs for a specified job definition.

Parameter Syntax

$result = $client->batchStopJobRun([
    'JobName' => '<string>', // REQUIRED
    'JobRunIds' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

The name of the job definition for which to stop job runs.

JobRunIds

Required: Yes
Type: Array of strings

A list of the JobRunIds that should be stopped for that job definition.

Result Syntax

[
    'Errors' => [
        [
            'ErrorDetail' => [
                'ErrorCode' => '<string>',
                'ErrorMessage' => '<string>',
            ],
            'JobName' => '<string>',
            'JobRunId' => '<string>',
        ],
        // ...
    ],
    'SuccessfulSubmissions' => [
        [
            'JobName' => '<string>',
            'JobRunId' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

Errors

Type: Array of BatchStopJobRunError structures

A list of the errors that were encountered in tryng to stop JobRuns, including the JobRunId for which each error was encountered and details about the error.

SuccessfulSubmissions

Type: Array of BatchStopJobRunSuccessfulSubmission structures

A list of the JobRuns that were successfully submitted for stopping.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

CreateClassifier

$result = $client->createClassifier([/* ... */]);
$promise = $client->createClassifierAsync([/* ... */]);

Creates a classifier in the user's account. This may be a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field of the request is present.

Parameter Syntax

$result = $client->createClassifier([
    'CsvClassifier' => [
        'AllowSingleColumn' => true || false,
        'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
        'Delimiter' => '<string>',
        'DisableValueTrimming' => true || false,
        'Header' => ['<string>', ...],
        'Name' => '<string>', // REQUIRED
        'QuoteSymbol' => '<string>',
    ],
    'GrokClassifier' => [
        'Classification' => '<string>', // REQUIRED
        'CustomPatterns' => '<string>',
        'GrokPattern' => '<string>', // REQUIRED
        'Name' => '<string>', // REQUIRED
    ],
    'JsonClassifier' => [
        'JsonPath' => '<string>', // REQUIRED
        'Name' => '<string>', // REQUIRED
    ],
    'XMLClassifier' => [
        'Classification' => '<string>', // REQUIRED
        'Name' => '<string>', // REQUIRED
        'RowTag' => '<string>',
    ],
]);

Parameter Details

Members

CsvClassifier

Type: CreateCsvClassifierRequest structure

A CsvClassifier object specifying the classifier to create.

GrokClassifier

Type: CreateGrokClassifierRequest structure

A GrokClassifier object specifying the classifier to create.

JsonClassifier

Type: CreateJsonClassifierRequest structure

A JsonClassifier object specifying the classifier to create.

XMLClassifier

Type: CreateXMLClassifierRequest structure

An XMLClassifier object specifying the classifier to create.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

AlreadyExistsException:
A resource to be created or added already exists.
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.

CreateConnection

$result = $client->createConnection([/* ... */]);
$promise = $client->createConnectionAsync([/* ... */]);

Creates a connection definition in the Data Catalog.

Parameter Syntax

$result = $client->createConnection([
    'CatalogId' => '<string>',
    'ConnectionInput' => [ // REQUIRED
        'ConnectionProperties' => ['<string>', ...], // REQUIRED
        'ConnectionType' => 'JDBC|SFTP', // REQUIRED
        'Description' => '<string>',
        'MatchCriteria' => ['<string>', ...],
        'Name' => '<string>', // REQUIRED
        'PhysicalConnectionRequirements' => [
            'AvailabilityZone' => '<string>',
            'SecurityGroupIdList' => ['<string>', ...],
            'SubnetId' => '<string>',
        ],
    ],
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which to create the connection. If none is provided, the AWS account ID is used by default.

ConnectionInput

Required: Yes
Type: ConnectionInput structure

A ConnectionInput object defining the connection to create.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

AlreadyExistsException:
A resource to be created or added already exists.
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
GlueEncryptionException:
An encryption operation failed.

CreateCrawler

$result = $client->createCrawler([/* ... */]);
$promise = $client->createCrawlerAsync([/* ... */]);

Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in the s3Targets field, the jdbcTargets field, or the DynamoDBTargets field.

Parameter Syntax

$result = $client->createCrawler([
    'Classifiers' => ['<string>', ...],
    'Configuration' => '<string>',
    'CrawlerSecurityConfiguration' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'Description' => '<string>',
    'Name' => '<string>', // REQUIRED
    'Role' => '<string>', // REQUIRED
    'Schedule' => '<string>',
    'SchemaChangePolicy' => [
        'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
        'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
    ],
    'TablePrefix' => '<string>',
    'Tags' => ['<string>', ...],
    'Targets' => [ // REQUIRED
        'DynamoDBTargets' => [
            [
                'Path' => '<string>',
            ],
            // ...
        ],
        'JdbcTargets' => [
            [
                'ConnectionName' => '<string>',
                'Exclusions' => ['<string>', ...],
                'Path' => '<string>',
            ],
            // ...
        ],
        'S3Targets' => [
            [
                'Exclusions' => ['<string>', ...],
                'Path' => '<string>',
            ],
            // ...
        ],
    ],
]);

Parameter Details

Members

Classifiers

Type: Array of strings

A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

Configuration

Type: string

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

CrawlerSecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used by this Crawler.

DatabaseName

Required: Yes
Type: string

The AWS Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/*.

Description

Type: string

A description of the new crawler.

Name

Required: Yes
Type: string

Name of the new crawler.

Role

Required: Yes
Type: string

The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

Schedule

Type: string

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

SchemaChangePolicy

Type: SchemaChangePolicy structure

Policy for the crawler's update and deletion behavior.

TablePrefix

Type: string

The table prefix used for catalog tables that are created.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

The tags to use with this crawler request. You may use tags to limit access to the crawler. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.

Targets

Required: Yes
Type: CrawlerTargets structure

A list of collection of targets to crawl.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
AlreadyExistsException:
A resource to be created or added already exists.
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.

CreateDatabase

$result = $client->createDatabase([/* ... */]);
$promise = $client->createDatabaseAsync([/* ... */]);

Creates a new database in a Data Catalog.

Parameter Syntax

$result = $client->createDatabase([
    'CatalogId' => '<string>',
    'DatabaseInput' => [ // REQUIRED
        'Description' => '<string>',
        'LocationUri' => '<string>',
        'Name' => '<string>', // REQUIRED
        'Parameters' => ['<string>', ...],
    ],
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which to create the database. If none is supplied, the AWS account ID is used by default.

DatabaseInput

Required: Yes
Type: DatabaseInput structure

A DatabaseInput object defining the metadata database to create in the catalog.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
AlreadyExistsException:
A resource to be created or added already exists.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

CreateDevEndpoint

$result = $client->createDevEndpoint([/* ... */]);
$promise = $client->createDevEndpointAsync([/* ... */]);

Creates a new DevEndpoint.

Parameter Syntax

$result = $client->createDevEndpoint([
    'Arguments' => ['<string>', ...],
    'EndpointName' => '<string>', // REQUIRED
    'ExtraJarsS3Path' => '<string>',
    'ExtraPythonLibsS3Path' => '<string>',
    'NumberOfNodes' => <integer>,
    'PublicKey' => '<string>',
    'PublicKeys' => ['<string>', ...],
    'RoleArn' => '<string>', // REQUIRED
    'SecurityConfiguration' => '<string>',
    'SecurityGroupIds' => ['<string>', ...],
    'SubnetId' => '<string>',
    'Tags' => ['<string>', ...],
]);

Parameter Details

Members

Arguments

Type: Associative array of custom strings keys (GenericString) to strings

A map of arguments used to configure the DevEndpoint.

EndpointName

Required: Yes
Type: string

The name to be assigned to the new DevEndpoint.

ExtraJarsS3Path

Type: string

Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.

ExtraPythonLibsS3Path

Type: string

Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

NumberOfNodes

Type: int

The number of AWS Glue Data Processing Units (DPUs) to allocate to this DevEndpoint.

PublicKey

Type: string

The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility, as the recommended attribute to use is public keys.

PublicKeys

Type: Array of strings

A list of public keys to be used by the DevEndpoints for authentication. The use of this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys: call the UpdateDevEndpoint API with the public key content in the deletePublicKeys attribute, and the list of new keys in the addPublicKeys attribute.

RoleArn

Required: Yes
Type: string

The IAM role for the DevEndpoint.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this DevEndpoint.

SecurityGroupIds

Type: Array of strings

Security group IDs for the security groups to be used by the new DevEndpoint.

SubnetId

Type: string

The subnet ID for the new DevEndpoint to use.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

The tags to use with this DevEndpoint. You may use tags to limit access to the DevEndpoint. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.

Result Syntax

[
    'Arguments' => ['<string>', ...],
    'AvailabilityZone' => '<string>',
    'CreatedTimestamp' => <DateTime>,
    'EndpointName' => '<string>',
    'ExtraJarsS3Path' => '<string>',
    'ExtraPythonLibsS3Path' => '<string>',
    'FailureReason' => '<string>',
    'NumberOfNodes' => <integer>,
    'RoleArn' => '<string>',
    'SecurityConfiguration' => '<string>',
    'SecurityGroupIds' => ['<string>', ...],
    'Status' => '<string>',
    'SubnetId' => '<string>',
    'VpcId' => '<string>',
    'YarnEndpointAddress' => '<string>',
    'ZeppelinRemoteSparkInterpreterPort' => <integer>,
]

Result Details

Members

Arguments

Type: Associative array of custom strings keys (GenericString) to strings

The map of arguments used to configure this DevEndpoint.

AvailabilityZone

Type: string

The AWS availability zone where this DevEndpoint is located.

CreatedTimestamp

Type: timestamp (string|DateTime or anything parsable by strtotime)

The point in time at which this DevEndpoint was created.

EndpointName

Type: string

The name assigned to the new DevEndpoint.

ExtraJarsS3Path

Type: string

Path to one or more Java Jars in an S3 bucket that will be loaded in your DevEndpoint.

ExtraPythonLibsS3Path

Type: string

Path(s) to one or more Python libraries in an S3 bucket that will be loaded in your DevEndpoint.

FailureReason

Type: string

The reason for a current failure in this DevEndpoint.

NumberOfNodes

Type: int

The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

RoleArn

Type: string

The AWS ARN of the role assigned to the new DevEndpoint.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure being used with this DevEndpoint.

SecurityGroupIds

Type: Array of strings

The security groups assigned to the new DevEndpoint.

Status

Type: string

The current status of the new DevEndpoint.

SubnetId

Type: string

The subnet ID assigned to the new DevEndpoint.

VpcId

Type: string

The ID of the VPC used by this DevEndpoint.

YarnEndpointAddress

Type: string

The address of the YARN endpoint used by this DevEndpoint.

ZeppelinRemoteSparkInterpreterPort

Type: int

The Apache Zeppelin port for the remote Apache Spark interpreter.

Errors

AccessDeniedException:
Access to a resource was denied.
AlreadyExistsException:
A resource to be created or added already exists.
IdempotentParameterMismatchException:
The same unique identifier was associated with two different records.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
ValidationException:
A value could not be validated.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.

CreateJob

$result = $client->createJob([/* ... */]);
$promise = $client->createJobAsync([/* ... */]);

Creates a new job definition.

Parameter Syntax

$result = $client->createJob([
    'AllocatedCapacity' => <integer>,
    'Command' => [ // REQUIRED
        'Name' => '<string>',
        'ScriptLocation' => '<string>',
    ],
    'Connections' => [
        'Connections' => ['<string>', ...],
    ],
    'DefaultArguments' => ['<string>', ...],
    'Description' => '<string>',
    'ExecutionProperty' => [
        'MaxConcurrentRuns' => <integer>,
    ],
    'LogUri' => '<string>',
    'MaxCapacity' => <float>,
    'MaxRetries' => <integer>,
    'Name' => '<string>', // REQUIRED
    'NotificationProperty' => [
        'NotifyDelayAfter' => <integer>,
    ],
    'NumberOfWorkers' => <integer>,
    'Role' => '<string>', // REQUIRED
    'SecurityConfiguration' => '<string>',
    'Tags' => ['<string>', ...],
    'Timeout' => <integer>,
    'WorkerType' => 'Standard|G.1X|G.2X',
]);

Parameter Details

Members

AllocatedCapacity

Type: int

This parameter is deprecated. Use MaxCapacity instead.

The number of AWS Glue data processing units (DPUs) to allocate to this Job. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.

Command

Required: Yes
Type: JobCommand structure

The JobCommand that executes this job.

Connections

Type: ConnectionsList structure

The connections used for this job.

DefaultArguments

Type: Associative array of custom strings keys (GenericString) to strings

The default arguments for this job.

You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.

For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.

For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

Description

Type: string

Description of the job being defined.

ExecutionProperty

Type: ExecutionProperty structure

An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job.

LogUri

Type: string

This field is reserved for future use.

MaxCapacity

Type: double

The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.

Do not set Max Capacity if using WorkerType and NumberOfWorkers.

The value that can be allocated for MaxCapacity depends on whether you are running a python shell job, or an Apache Spark ETL job:

When you specify a python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
When you specify an Apache Spark ETL job (JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

MaxRetries

Type: int

The maximum number of times to retry this job if it fails.

Name

Required: Yes
Type: string

The name you assign to this job definition. It must be unique in your account.

NotificationProperty

Type: NotificationProperty structure

Specifies configuration properties of a job notification.

NumberOfWorkers

Type: int

The number of workers of a defined workerType that are allocated when a job runs.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

Role

Required: Yes
Type: string

The name or ARN of the IAM role associated with this job.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this job.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

The tags to use with this job. You may use tags to limit access to the job. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.

Timeout

Type: int

The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

WorkerType

Type: string

The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Members

Name

Type: string

The unique name that was provided for this job definition.

Errors

InvalidInputException:
The input provided was not valid.
IdempotentParameterMismatchException:
The same unique identifier was associated with two different records.
AlreadyExistsException:
A resource to be created or added already exists.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.

CreatePartition

$result = $client->createPartition([/* ... */]);
$promise = $client->createPartitionAsync([/* ... */]);

Creates a new partition.

Parameter Syntax

$result = $client->createPartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionInput' => [ // REQUIRED
        'LastAccessTime' => <integer || string || DateTime>,
        'LastAnalyzedTime' => <integer || string || DateTime>,
        'Parameters' => ['<string>', ...],
        'StorageDescriptor' => [
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>', // REQUIRED
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>', // REQUIRED
                    'SortOrder' => <integer>, // REQUIRED
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'Values' => ['<string>', ...],
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the catalog in which the partion is to be created. Currently, this should be the AWS account ID.

DatabaseName

Required: Yes
Type: string

The name of the metadata database in which the partition is to be created.

PartitionInput

Required: Yes
Type: PartitionInput structure

A PartitionInput structure defining the partition to be created.

TableName

Required: Yes
Type: string

The name of the metadata table in which the partition is to be created.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
AlreadyExistsException:
A resource to be created or added already exists.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
InternalServiceException:
An internal service error occurred.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

CreateScript

$result = $client->createScript([/* ... */]);
$promise = $client->createScriptAsync([/* ... */]);

Transforms a directed acyclic graph (DAG) into code.

Parameter Syntax

$result = $client->createScript([
    'DagEdges' => [
        [
            'Source' => '<string>', // REQUIRED
            'Target' => '<string>', // REQUIRED
            'TargetParameter' => '<string>',
        ],
        // ...
    ],
    'DagNodes' => [
        [
            'Args' => [ // REQUIRED
                [
                    'Name' => '<string>', // REQUIRED
                    'Param' => true || false,
                    'Value' => '<string>', // REQUIRED
                ],
                // ...
            ],
            'Id' => '<string>', // REQUIRED
            'LineNumber' => <integer>,
            'NodeType' => '<string>', // REQUIRED
        ],
        // ...
    ],
    'Language' => 'PYTHON|SCALA',
]);

Parameter Details

Members

DagEdges

Type: Array of CodeGenEdge structures

A list of the edges in the DAG.

DagNodes

Type: Array of CodeGenNode structures

A list of the nodes in the DAG.

Language

Type: string

The programming language of the resulting code from the DAG.

Result Syntax

[
    'PythonScript' => '<string>',
    'ScalaCode' => '<string>',
]

Result Details

Members

PythonScript

Type: string

The Python script generated from the DAG.

ScalaCode

Type: string

The Scala code generated from the DAG.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

CreateSecurityConfiguration

$result = $client->createSecurityConfiguration([/* ... */]);
$promise = $client->createSecurityConfigurationAsync([/* ... */]);

Creates a new security configuration.

Parameter Syntax

$result = $client->createSecurityConfiguration([
    'EncryptionConfiguration' => [ // REQUIRED
        'CloudWatchEncryption' => [
            'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
            'KmsKeyArn' => '<string>',
        ],
        'JobBookmarksEncryption' => [
            'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
            'KmsKeyArn' => '<string>',
        ],
        'S3Encryption' => [
            [
                'KmsKeyArn' => '<string>',
                'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
            ],
            // ...
        ],
    ],
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

EncryptionConfiguration

Required: Yes
Type: EncryptionConfiguration structure

The encryption configuration for the new security configuration.

Name

Required: Yes
Type: string

The name for the new security configuration.

Result Syntax

[
    'CreatedTimestamp' => <DateTime>,
    'Name' => '<string>',
]

Result Details

Members

CreatedTimestamp

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time at which the new security configuration was created.

Name

Type: string

The name assigned to the new security configuration.

Errors

AlreadyExistsException:
A resource to be created or added already exists.
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.

CreateTable

$result = $client->createTable([/* ... */]);
$promise = $client->createTableAsync([/* ... */]);

Creates a new table definition in the Data Catalog.

Parameter Syntax

$result = $client->createTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TableInput' => [ // REQUIRED
        'Description' => '<string>',
        'LastAccessTime' => <integer || string || DateTime>,
        'LastAnalyzedTime' => <integer || string || DateTime>,
        'Name' => '<string>', // REQUIRED
        'Owner' => '<string>',
        'Parameters' => ['<string>', ...],
        'PartitionKeys' => [
            [
                'Comment' => '<string>',
                'Name' => '<string>', // REQUIRED
                'Type' => '<string>',
            ],
            // ...
        ],
        'Retention' => <integer>,
        'StorageDescriptor' => [
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>', // REQUIRED
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>', // REQUIRED
                    'SortOrder' => <integer>, // REQUIRED
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'TableType' => '<string>',
        'ViewExpandedText' => '<string>',
        'ViewOriginalText' => '<string>',
    ],
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which to create the Table. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The catalog database in which to create the new table. For Hive compatibility, this name is entirely lowercase.

TableInput

Required: Yes
Type: TableInput structure

The TableInput object that defines the metadata table to create in the catalog.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

AlreadyExistsException:
A resource to be created or added already exists.
InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

CreateTrigger

$result = $client->createTrigger([/* ... */]);
$promise = $client->createTriggerAsync([/* ... */]);

Creates a new trigger.

Parameter Syntax

$result = $client->createTrigger([
    'Actions' => [ // REQUIRED
        [
            'Arguments' => ['<string>', ...],
            'JobName' => '<string>',
            'NotificationProperty' => [
                'NotifyDelayAfter' => <integer>,
            ],
            'SecurityConfiguration' => '<string>',
            'Timeout' => <integer>,
        ],
        // ...
    ],
    'Description' => '<string>',
    'Name' => '<string>', // REQUIRED
    'Predicate' => [
        'Conditions' => [
            [
                'JobName' => '<string>',
                'LogicalOperator' => 'EQUALS',
                'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
            ],
            // ...
        ],
        'Logical' => 'AND|ANY',
    ],
    'Schedule' => '<string>',
    'StartOnCreation' => true || false,
    'Tags' => ['<string>', ...],
    'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND', // REQUIRED
]);

Parameter Details

Members

Actions

Required: Yes
Type: Array of Action structures

The actions initiated by this trigger when it fires.

Description

Type: string

A description of the new trigger.

Name

Required: Yes
Type: string

The name of the trigger.

Predicate

Type: Predicate structure

A predicate to specify when the new trigger should fire.

This field is required when the trigger type is CONDITIONAL.

Schedule

Type: string

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

This field is required when the trigger type is SCHEDULED.

StartOnCreation

Type: boolean

Set to true to start SCHEDULED and CONDITIONAL triggers when created. True not supported for ON_DEMAND triggers.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

The tags to use with this trigger. You may use tags to limit access to the trigger. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.

Type

Required: Yes
Type: string

The type of the new trigger.

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Members

Name

Type: string

The name of the trigger.

Errors

AlreadyExistsException:
A resource to be created or added already exists.
InvalidInputException:
The input provided was not valid.
IdempotentParameterMismatchException:
The same unique identifier was associated with two different records.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.

CreateUserDefinedFunction

$result = $client->createUserDefinedFunction([/* ... */]);
$promise = $client->createUserDefinedFunctionAsync([/* ... */]);

Creates a new function definition in the Data Catalog.

Parameter Syntax

$result = $client->createUserDefinedFunction([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'FunctionInput' => [ // REQUIRED
        'ClassName' => '<string>',
        'FunctionName' => '<string>',
        'OwnerName' => '<string>',
        'OwnerType' => 'USER|ROLE|GROUP',
        'ResourceUris' => [
            [
                'ResourceType' => 'JAR|FILE|ARCHIVE',
                'Uri' => '<string>',
            ],
            // ...
        ],
    ],
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which to create the function. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database in which to create the function.

FunctionInput

Required: Yes
Type: UserDefinedFunctionInput structure

A FunctionInput object that defines the function to create in the Data Catalog.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

AlreadyExistsException:
A resource to be created or added already exists.
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
GlueEncryptionException:
An encryption operation failed.

DeleteClassifier

$result = $client->deleteClassifier([/* ... */]);
$promise = $client->deleteClassifierAsync([/* ... */]);

Removes a classifier from the Data Catalog.

Parameter Syntax

$result = $client->deleteClassifier([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

Name of the classifier to remove.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.

DeleteConnection

$result = $client->deleteConnection([/* ... */]);
$promise = $client->deleteConnectionAsync([/* ... */]);

Deletes a connection from the Data Catalog.

Parameter Syntax

$result = $client->deleteConnection([
    'CatalogId' => '<string>',
    'ConnectionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default.

ConnectionName

Required: Yes
Type: string

The name of the connection to delete.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.

DeleteCrawler

$result = $client->deleteCrawler([/* ... */]);
$promise = $client->deleteCrawlerAsync([/* ... */]);

Removes a specified crawler from the Data Catalog, unless the crawler state is RUNNING.

Parameter Syntax

$result = $client->deleteCrawler([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

Name of the crawler to remove.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
CrawlerRunningException:
The operation cannot be performed because the crawler is already running.
SchedulerTransitioningException:
The specified scheduler is transitioning.
OperationTimeoutException:
The operation timed out.

DeleteDatabase

$result = $client->deleteDatabase([/* ... */]);
$promise = $client->deleteDatabaseAsync([/* ... */]);

Removes a specified Database from a Data Catalog.

After completing this operation, you will no longer have access to the tables (and all table versions and partitions that might belong to the tables) and the user-defined functions in the deleted database. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.

To ensure immediate deletion of all related resources, before calling DeleteDatabase, use DeleteTableVersion or BatchDeleteTableVersion, DeletePartition or BatchDeletePartition, DeleteUserDefinedFunction, and DeleteTable or BatchDeleteTable, to delete any resources that belong to the database.

Parameter Syntax

$result = $client->deleteDatabase([
    'CatalogId' => '<string>',
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the database resides. If none is supplied, the AWS account ID is used by default.

Name

Required: Yes
Type: string

The name of the Database to delete. For Hive compatibility, this must be all lowercase.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

DeleteDevEndpoint

$result = $client->deleteDevEndpoint([/* ... */]);
$promise = $client->deleteDevEndpointAsync([/* ... */]);

Deletes a specified DevEndpoint.

Parameter Syntax

$result = $client->deleteDevEndpoint([
    'EndpointName' => '<string>', // REQUIRED
]);

Parameter Details

Members

EndpointName

Required: Yes
Type: string

The name of the DevEndpoint.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

DeleteJob

$result = $client->deleteJob([/* ... */]);
$promise = $client->deleteJobAsync([/* ... */]);

Deletes a specified job definition. If the job definition is not found, no exception is thrown.

Parameter Syntax

$result = $client->deleteJob([
    'JobName' => '<string>', // REQUIRED
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

The name of the job definition to delete.

Result Syntax

[
    'JobName' => '<string>',
]

Result Details

Members

JobName

Type: string

The name of the job definition that was deleted.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

DeletePartition

$result = $client->deletePartition([/* ... */]);
$promise = $client->deletePartitionAsync([/* ... */]);

Deletes a specified partition.

Parameter Syntax

$result = $client->deletePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionValues' => ['<string>', ...], // REQUIRED
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the partition to be deleted resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database in which the table in question resides.

PartitionValues

Required: Yes
Type: Array of strings

The values that define the partition.

TableName

Required: Yes
Type: string

The name of the table where the partition to be deleted is located.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

DeleteResourcePolicy

$result = $client->deleteResourcePolicy([/* ... */]);
$promise = $client->deleteResourcePolicyAsync([/* ... */]);

Deletes a specified policy.

Parameter Syntax

$result = $client->deleteResourcePolicy([
    'PolicyHashCondition' => '<string>',
]);

Parameter Details

Members

PolicyHashCondition

Type: string

The hash value returned when this policy was set.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
ConditionCheckFailureException:
A specified condition was not satisfied.

DeleteSecurityConfiguration

$result = $client->deleteSecurityConfiguration([/* ... */]);
$promise = $client->deleteSecurityConfigurationAsync([/* ... */]);

Deletes a specified security configuration.

Parameter Syntax

$result = $client->deleteSecurityConfiguration([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the security configuration to delete.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

DeleteTable

$result = $client->deleteTable([/* ... */]);
$promise = $client->deleteTableAsync([/* ... */]);

Removes a table definition from the Data Catalog.

To ensure immediate deletion of all related resources, before calling DeleteTable, use DeleteTableVersion or BatchDeleteTableVersion, and DeletePartition or BatchDeletePartition, to delete any resources that belong to the table.

Parameter Syntax

$result = $client->deleteTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database in which the table resides. For Hive compatibility, this name is entirely lowercase.

Name

Required: Yes
Type: string

The name of the table to be deleted. For Hive compatibility, this name is entirely lowercase.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

DeleteTableVersion

$result = $client->deleteTableVersion([/* ... */]);
$promise = $client->deleteTableVersionAsync([/* ... */]);

Deletes a specified version of a table.

Parameter Syntax

$result = $client->deleteTableVersion([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TableName' => '<string>', // REQUIRED
    'VersionId' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.

TableName

Required: Yes
Type: string

The name of the table. For Hive compatibility, this name is entirely lowercase.

VersionId

Required: Yes
Type: string

The ID of the table version to be deleted. A VersionID is a string representation of an integer. Each version is incremented by 1.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

DeleteTrigger

$result = $client->deleteTrigger([/* ... */]);
$promise = $client->deleteTriggerAsync([/* ... */]);

Deletes a specified trigger. If the trigger is not found, no exception is thrown.

Parameter Syntax

$result = $client->deleteTrigger([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the trigger to delete.

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Members

Name

Type: string

The name of the trigger that was deleted.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.

DeleteUserDefinedFunction

$result = $client->deleteUserDefinedFunction([/* ... */]);
$promise = $client->deleteUserDefinedFunctionAsync([/* ... */]);

Deletes an existing function definition from the Data Catalog.

Parameter Syntax

$result = $client->deleteUserDefinedFunction([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'FunctionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the function to be deleted is located. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the function is located.

FunctionName

Required: Yes
Type: string

The name of the function definition to be deleted.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetCatalogImportStatus

$result = $client->getCatalogImportStatus([/* ... */]);
$promise = $client->getCatalogImportStatusAsync([/* ... */]);

Retrieves the status of a migration operation.

Parameter Syntax

$result = $client->getCatalogImportStatus([
    'CatalogId' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the catalog to migrate. Currently, this should be the AWS account ID.

Result Syntax

[
    'ImportStatus' => [
        'ImportCompleted' => true || false,
        'ImportTime' => <DateTime>,
        'ImportedBy' => '<string>',
    ],
]

Result Details

Members

ImportStatus

Type: CatalogImportStatus structure

The status of the specified catalog migration.

Errors

InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetClassifier

$result = $client->getClassifier([/* ... */]);
$promise = $client->getClassifierAsync([/* ... */]);

Retrieve a classifier by name.

Parameter Syntax

$result = $client->getClassifier([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

Name of the classifier to retrieve.

Result Syntax

[
    'Classifier' => [
        'CsvClassifier' => [
            'AllowSingleColumn' => true || false,
            'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
            'CreationTime' => <DateTime>,
            'Delimiter' => '<string>',
            'DisableValueTrimming' => true || false,
            'Header' => ['<string>', ...],
            'LastUpdated' => <DateTime>,
            'Name' => '<string>',
            'QuoteSymbol' => '<string>',
            'Version' => <integer>,
        ],
        'GrokClassifier' => [
            'Classification' => '<string>',
            'CreationTime' => <DateTime>,
            'CustomPatterns' => '<string>',
            'GrokPattern' => '<string>',
            'LastUpdated' => <DateTime>,
            'Name' => '<string>',
            'Version' => <integer>,
        ],
        'JsonClassifier' => [
            'CreationTime' => <DateTime>,
            'JsonPath' => '<string>',
            'LastUpdated' => <DateTime>,
            'Name' => '<string>',
            'Version' => <integer>,
        ],
        'XMLClassifier' => [
            'Classification' => '<string>',
            'CreationTime' => <DateTime>,
            'LastUpdated' => <DateTime>,
            'Name' => '<string>',
            'RowTag' => '<string>',
            'Version' => <integer>,
        ],
    ],
]

Result Details

Members

Classifier

Type: Classifier structure

The requested classifier.

Errors

EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.

GetClassifiers

$result = $client->getClassifiers([/* ... */]);
$promise = $client->getClassifiersAsync([/* ... */]);

Lists all classifier objects in the Data Catalog.

Parameter Syntax

$result = $client->getClassifiers([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

MaxResults

Type: int

Size of the list to return (optional).

NextToken

Type: string

An optional continuation token.

Result Syntax

[
    'Classifiers' => [
        [
            'CsvClassifier' => [
                'AllowSingleColumn' => true || false,
                'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
                'CreationTime' => <DateTime>,
                'Delimiter' => '<string>',
                'DisableValueTrimming' => true || false,
                'Header' => ['<string>', ...],
                'LastUpdated' => <DateTime>,
                'Name' => '<string>',
                'QuoteSymbol' => '<string>',
                'Version' => <integer>,
            ],
            'GrokClassifier' => [
                'Classification' => '<string>',
                'CreationTime' => <DateTime>,
                'CustomPatterns' => '<string>',
                'GrokPattern' => '<string>',
                'LastUpdated' => <DateTime>,
                'Name' => '<string>',
                'Version' => <integer>,
            ],
            'JsonClassifier' => [
                'CreationTime' => <DateTime>,
                'JsonPath' => '<string>',
                'LastUpdated' => <DateTime>,
                'Name' => '<string>',
                'Version' => <integer>,
            ],
            'XMLClassifier' => [
                'Classification' => '<string>',
                'CreationTime' => <DateTime>,
                'LastUpdated' => <DateTime>,
                'Name' => '<string>',
                'RowTag' => '<string>',
                'Version' => <integer>,
            ],
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

Classifiers

Type: Array of Classifier structures

The requested list of classifier objects.

NextToken

Type: string

A continuation token.

Errors

OperationTimeoutException:
The operation timed out.

GetConnection

$result = $client->getConnection([/* ... */]);
$promise = $client->getConnectionAsync([/* ... */]);

Retrieves a connection definition from the Data Catalog.

Parameter Syntax

$result = $client->getConnection([
    'CatalogId' => '<string>',
    'HidePassword' => true || false,
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default.

HidePassword

Type: boolean

Allows you to retrieve the connection metadata without returning the password. For instance, the AWS Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the AWS KMS key to decrypt the password, but does have permission to access the rest of the connection properties.

Name

Required: Yes
Type: string

The name of the connection definition to retrieve.

Result Syntax

[
    'Connection' => [
        'ConnectionProperties' => ['<string>', ...],
        'ConnectionType' => 'JDBC|SFTP',
        'CreationTime' => <DateTime>,
        'Description' => '<string>',
        'LastUpdatedBy' => '<string>',
        'LastUpdatedTime' => <DateTime>,
        'MatchCriteria' => ['<string>', ...],
        'Name' => '<string>',
        'PhysicalConnectionRequirements' => [
            'AvailabilityZone' => '<string>',
            'SecurityGroupIdList' => ['<string>', ...],
            'SubnetId' => '<string>',
        ],
    ],
]

Result Details

Members

Connection

Type: Connection structure

The requested connection definition.

Errors

EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
GlueEncryptionException:
An encryption operation failed.

GetConnections

$result = $client->getConnections([/* ... */]);
$promise = $client->getConnectionsAsync([/* ... */]);

Retrieves a list of connection definitions from the Data Catalog.

Parameter Syntax

$result = $client->getConnections([
    'CatalogId' => '<string>',
    'Filter' => [
        'ConnectionType' => 'JDBC|SFTP',
        'MatchCriteria' => ['<string>', ...],
    ],
    'HidePassword' => true || false,
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the connections reside. If none is provided, the AWS account ID is used by default.

Filter

Type: GetConnectionsFilter structure

A filter that controls which connections will be returned.

HidePassword

Type: boolean

MaxResults

Type: int

The maximum number of connections to return in one response.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'ConnectionList' => [
        [
            'ConnectionProperties' => ['<string>', ...],
            'ConnectionType' => 'JDBC|SFTP',
            'CreationTime' => <DateTime>,
            'Description' => '<string>',
            'LastUpdatedBy' => '<string>',
            'LastUpdatedTime' => <DateTime>,
            'MatchCriteria' => ['<string>', ...],
            'Name' => '<string>',
            'PhysicalConnectionRequirements' => [
                'AvailabilityZone' => '<string>',
                'SecurityGroupIdList' => ['<string>', ...],
                'SubnetId' => '<string>',
            ],
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

ConnectionList

Type: Array of Connection structures

A list of requested connection definitions.

NextToken

Type: string

A continuation token, if the list of connections returned does not include the last of the filtered connections.

Errors

EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
GlueEncryptionException:
An encryption operation failed.

GetCrawler

$result = $client->getCrawler([/* ... */]);
$promise = $client->getCrawlerAsync([/* ... */]);

Retrieves metadata for a specified crawler.

Parameter Syntax

$result = $client->getCrawler([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

Name of the crawler to retrieve metadata for.

Result Syntax

[
    'Crawler' => [
        'Classifiers' => ['<string>', ...],
        'Configuration' => '<string>',
        'CrawlElapsedTime' => <integer>,
        'CrawlerSecurityConfiguration' => '<string>',
        'CreationTime' => <DateTime>,
        'DatabaseName' => '<string>',
        'Description' => '<string>',
        'LastCrawl' => [
            'ErrorMessage' => '<string>',
            'LogGroup' => '<string>',
            'LogStream' => '<string>',
            'MessagePrefix' => '<string>',
            'StartTime' => <DateTime>,
            'Status' => 'SUCCEEDED|CANCELLED|FAILED',
        ],
        'LastUpdated' => <DateTime>,
        'Name' => '<string>',
        'Role' => '<string>',
        'Schedule' => [
            'ScheduleExpression' => '<string>',
            'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
        ],
        'SchemaChangePolicy' => [
            'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
            'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
        ],
        'State' => 'READY|RUNNING|STOPPING',
        'TablePrefix' => '<string>',
        'Targets' => [
            'DynamoDBTargets' => [
                [
                    'Path' => '<string>',
                ],
                // ...
            ],
            'JdbcTargets' => [
                [
                    'ConnectionName' => '<string>',
                    'Exclusions' => ['<string>', ...],
                    'Path' => '<string>',
                ],
                // ...
            ],
            'S3Targets' => [
                [
                    'Exclusions' => ['<string>', ...],
                    'Path' => '<string>',
                ],
                // ...
            ],
        ],
        'Version' => <integer>,
    ],
]

Result Details

Members

Crawler

Type: Crawler structure

The metadata for the specified crawler.

Errors

EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.

GetCrawlerMetrics

$result = $client->getCrawlerMetrics([/* ... */]);
$promise = $client->getCrawlerMetricsAsync([/* ... */]);

Retrieves metrics about specified crawlers.

Parameter Syntax

$result = $client->getCrawlerMetrics([
    'CrawlerNameList' => ['<string>', ...],
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

CrawlerNameList

Type: Array of strings

A list of the names of crawlers about which to retrieve metrics.

MaxResults

Type: int

The maximum size of a list to return.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'CrawlerMetricsList' => [
        [
            'CrawlerName' => '<string>',
            'LastRuntimeSeconds' => <float>,
            'MedianRuntimeSeconds' => <float>,
            'StillEstimating' => true || false,
            'TablesCreated' => <integer>,
            'TablesDeleted' => <integer>,
            'TablesUpdated' => <integer>,
            'TimeLeftSeconds' => <float>,
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

CrawlerMetricsList

Type: Array of CrawlerMetrics structures

A list of metrics for the specified crawler.

NextToken

Type: string

A continuation token, if the returned list does not contain the last metric available.

Errors

OperationTimeoutException:
The operation timed out.

GetCrawlers

$result = $client->getCrawlers([/* ... */]);
$promise = $client->getCrawlersAsync([/* ... */]);

Retrieves metadata for all crawlers defined in the customer account.

Parameter Syntax

$result = $client->getCrawlers([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

MaxResults

Type: int

The number of crawlers to return on each call.

NextToken

Type: string

A continuation token, if this is a continuation request.

Result Syntax

[
    'Crawlers' => [
        [
            'Classifiers' => ['<string>', ...],
            'Configuration' => '<string>',
            'CrawlElapsedTime' => <integer>,
            'CrawlerSecurityConfiguration' => '<string>',
            'CreationTime' => <DateTime>,
            'DatabaseName' => '<string>',
            'Description' => '<string>',
            'LastCrawl' => [
                'ErrorMessage' => '<string>',
                'LogGroup' => '<string>',
                'LogStream' => '<string>',
                'MessagePrefix' => '<string>',
                'StartTime' => <DateTime>,
                'Status' => 'SUCCEEDED|CANCELLED|FAILED',
            ],
            'LastUpdated' => <DateTime>,
            'Name' => '<string>',
            'Role' => '<string>',
            'Schedule' => [
                'ScheduleExpression' => '<string>',
                'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
            ],
            'SchemaChangePolicy' => [
                'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
                'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
            ],
            'State' => 'READY|RUNNING|STOPPING',
            'TablePrefix' => '<string>',
            'Targets' => [
                'DynamoDBTargets' => [
                    [
                        'Path' => '<string>',
                    ],
                    // ...
                ],
                'JdbcTargets' => [
                    [
                        'ConnectionName' => '<string>',
                        'Exclusions' => ['<string>', ...],
                        'Path' => '<string>',
                    ],
                    // ...
                ],
                'S3Targets' => [
                    [
                        'Exclusions' => ['<string>', ...],
                        'Path' => '<string>',
                    ],
                    // ...
                ],
            ],
            'Version' => <integer>,
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

Crawlers

Type: Array of Crawler structures

A list of crawler metadata.

NextToken

Type: string

A continuation token, if the returned list has not reached the end of those defined in this customer account.

Errors

OperationTimeoutException:
The operation timed out.

GetDataCatalogEncryptionSettings

$result = $client->getDataCatalogEncryptionSettings([/* ... */]);
$promise = $client->getDataCatalogEncryptionSettingsAsync([/* ... */]);

Retrieves the security configuration for a specified catalog.

Parameter Syntax

$result = $client->getDataCatalogEncryptionSettings([
    'CatalogId' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog for which to retrieve the security configuration. If none is provided, the AWS account ID is used by default.

Result Syntax

[
    'DataCatalogEncryptionSettings' => [
        'ConnectionPasswordEncryption' => [
            'AwsKmsKeyId' => '<string>',
            'ReturnConnectionPasswordEncrypted' => true || false,
        ],
        'EncryptionAtRest' => [
            'CatalogEncryptionMode' => 'DISABLED|SSE-KMS',
            'SseAwsKmsKeyId' => '<string>',
        ],
    ],
]

Result Details

Members

DataCatalogEncryptionSettings

Type: DataCatalogEncryptionSettings structure

The requested security configuration.

Errors

InternalServiceException:
An internal service error occurred.
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.

GetDatabase

$result = $client->getDatabase([/* ... */]);
$promise = $client->getDatabaseAsync([/* ... */]);

Retrieves the definition of a specified database.

Parameter Syntax

$result = $client->getDatabase([
    'CatalogId' => '<string>',
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the database resides. If none is supplied, the AWS account ID is used by default.

Name

Required: Yes
Type: string

The name of the database to retrieve. For Hive compatibility, this should be all lowercase.

Result Syntax

[
    'Database' => [
        'CreateTime' => <DateTime>,
        'Description' => '<string>',
        'LocationUri' => '<string>',
        'Name' => '<string>',
        'Parameters' => ['<string>', ...],
    ],
]

Result Details

Members

Database

Type: Database structure

The definition of the specified database in the catalog.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetDatabases

$result = $client->getDatabases([/* ... */]);
$promise = $client->getDatabasesAsync([/* ... */]);

Retrieves all Databases defined in a given Data Catalog.

Parameter Syntax

$result = $client->getDatabases([
    'CatalogId' => '<string>',
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog from which to retrieve Databases. If none is supplied, the AWS account ID is used by default.

MaxResults

Type: int

The maximum number of databases to return in one response.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'DatabaseList' => [
        [
            'CreateTime' => <DateTime>,
            'Description' => '<string>',
            'LocationUri' => '<string>',
            'Name' => '<string>',
            'Parameters' => ['<string>', ...],
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

DatabaseList

Required: Yes
Type: Array of Database structures

A list of Database objects from the specified catalog.

NextToken

Type: string

A continuation token for paginating the returned list of tokens, returned if the current segment of the list is not the last.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetDataflowGraph

$result = $client->getDataflowGraph([/* ... */]);
$promise = $client->getDataflowGraphAsync([/* ... */]);

Transforms a Python script into a directed acyclic graph (DAG).

Parameter Syntax

$result = $client->getDataflowGraph([
    'PythonScript' => '<string>',
]);

Parameter Details

Members

PythonScript

Type: string

The Python script to transform.

Result Syntax

[
    'DagEdges' => [
        [
            'Source' => '<string>',
            'Target' => '<string>',
            'TargetParameter' => '<string>',
        ],
        // ...
    ],
    'DagNodes' => [
        [
            'Args' => [
                [
                    'Name' => '<string>',
                    'Param' => true || false,
                    'Value' => '<string>',
                ],
                // ...
            ],
            'Id' => '<string>',
            'LineNumber' => <integer>,
            'NodeType' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

DagEdges

Type: Array of CodeGenEdge structures

A list of the edges in the resulting DAG.

DagNodes

Type: Array of CodeGenNode structures

A list of the nodes in the resulting DAG.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetDevEndpoint

$result = $client->getDevEndpoint([/* ... */]);
$promise = $client->getDevEndpointAsync([/* ... */]);

Retrieves information about a specified DevEndpoint.

When you create a development endpoint in a virtual private cloud (VPC), AWS Glue returns only a private IP address, and the public IP address field is not populated. When you create a non-VPC development endpoint, AWS Glue returns only a public IP address.

Parameter Syntax

$result = $client->getDevEndpoint([
    'EndpointName' => '<string>', // REQUIRED
]);

Parameter Details

Members

EndpointName

Required: Yes
Type: string

Name of the DevEndpoint for which to retrieve information.

Result Syntax

[
    'DevEndpoint' => [
        'Arguments' => ['<string>', ...],
        'AvailabilityZone' => '<string>',
        'CreatedTimestamp' => <DateTime>,
        'EndpointName' => '<string>',
        'ExtraJarsS3Path' => '<string>',
        'ExtraPythonLibsS3Path' => '<string>',
        'FailureReason' => '<string>',
        'LastModifiedTimestamp' => <DateTime>,
        'LastUpdateStatus' => '<string>',
        'NumberOfNodes' => <integer>,
        'PrivateAddress' => '<string>',
        'PublicAddress' => '<string>',
        'PublicKey' => '<string>',
        'PublicKeys' => ['<string>', ...],
        'RoleArn' => '<string>',
        'SecurityConfiguration' => '<string>',
        'SecurityGroupIds' => ['<string>', ...],
        'Status' => '<string>',
        'SubnetId' => '<string>',
        'VpcId' => '<string>',
        'YarnEndpointAddress' => '<string>',
        'ZeppelinRemoteSparkInterpreterPort' => <integer>,
    ],
]

Result Details

Members

DevEndpoint

Type: DevEndpoint structure

A DevEndpoint definition.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

GetDevEndpoints

$result = $client->getDevEndpoints([/* ... */]);
$promise = $client->getDevEndpointsAsync([/* ... */]);

Retrieves all the DevEndpoints in this AWS account.

When you create a development endpoint in a virtual private cloud (VPC), AWS Glue returns only a private IP address and the public IP address field is not populated. When you create a non-VPC development endpoint, AWS Glue returns only a public IP address.

Parameter Syntax

$result = $client->getDevEndpoints([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

MaxResults

Type: int

The maximum size of information to return.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'DevEndpoints' => [
        [
            'Arguments' => ['<string>', ...],
            'AvailabilityZone' => '<string>',
            'CreatedTimestamp' => <DateTime>,
            'EndpointName' => '<string>',
            'ExtraJarsS3Path' => '<string>',
            'ExtraPythonLibsS3Path' => '<string>',
            'FailureReason' => '<string>',
            'LastModifiedTimestamp' => <DateTime>,
            'LastUpdateStatus' => '<string>',
            'NumberOfNodes' => <integer>,
            'PrivateAddress' => '<string>',
            'PublicAddress' => '<string>',
            'PublicKey' => '<string>',
            'PublicKeys' => ['<string>', ...],
            'RoleArn' => '<string>',
            'SecurityConfiguration' => '<string>',
            'SecurityGroupIds' => ['<string>', ...],
            'Status' => '<string>',
            'SubnetId' => '<string>',
            'VpcId' => '<string>',
            'YarnEndpointAddress' => '<string>',
            'ZeppelinRemoteSparkInterpreterPort' => <integer>,
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

DevEndpoints

Type: Array of DevEndpoint structures

A list of DevEndpoint definitions.

NextToken

Type: string

A continuation token, if not all DevEndpoint definitions have yet been returned.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

GetJob

$result = $client->getJob([/* ... */]);
$promise = $client->getJobAsync([/* ... */]);

Retrieves an existing job definition.

Parameter Syntax

$result = $client->getJob([
    'JobName' => '<string>', // REQUIRED
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

The name of the job definition to retrieve.

Result Syntax

[
    'Job' => [
        'AllocatedCapacity' => <integer>,
        'Command' => [
            'Name' => '<string>',
            'ScriptLocation' => '<string>',
        ],
        'Connections' => [
            'Connections' => ['<string>', ...],
        ],
        'CreatedOn' => <DateTime>,
        'DefaultArguments' => ['<string>', ...],
        'Description' => '<string>',
        'ExecutionProperty' => [
            'MaxConcurrentRuns' => <integer>,
        ],
        'LastModifiedOn' => <DateTime>,
        'LogUri' => '<string>',
        'MaxCapacity' => <float>,
        'MaxRetries' => <integer>,
        'Name' => '<string>',
        'NotificationProperty' => [
            'NotifyDelayAfter' => <integer>,
        ],
        'NumberOfWorkers' => <integer>,
        'Role' => '<string>',
        'SecurityConfiguration' => '<string>',
        'Timeout' => <integer>,
        'WorkerType' => 'Standard|G.1X|G.2X',
    ],
]

Result Details

Members

Job

Type: Job structure

The requested job definition.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetJobRun

$result = $client->getJobRun([/* ... */]);
$promise = $client->getJobRunAsync([/* ... */]);

Retrieves the metadata for a given job run.

Parameter Syntax

$result = $client->getJobRun([
    'JobName' => '<string>', // REQUIRED
    'PredecessorsIncluded' => true || false,
    'RunId' => '<string>', // REQUIRED
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

Name of the job definition being run.

PredecessorsIncluded

Type: boolean

True if a list of predecessor runs should be returned.

RunId

Required: Yes
Type: string

The ID of the job run.

Result Syntax

[
    'JobRun' => [
        'AllocatedCapacity' => <integer>,
        'Arguments' => ['<string>', ...],
        'Attempt' => <integer>,
        'CompletedOn' => <DateTime>,
        'ErrorMessage' => '<string>',
        'ExecutionTime' => <integer>,
        'Id' => '<string>',
        'JobName' => '<string>',
        'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
        'LastModifiedOn' => <DateTime>,
        'LogGroupName' => '<string>',
        'MaxCapacity' => <float>,
        'NotificationProperty' => [
            'NotifyDelayAfter' => <integer>,
        ],
        'NumberOfWorkers' => <integer>,
        'PredecessorRuns' => [
            [
                'JobName' => '<string>',
                'RunId' => '<string>',
            ],
            // ...
        ],
        'PreviousRunId' => '<string>',
        'SecurityConfiguration' => '<string>',
        'StartedOn' => <DateTime>,
        'Timeout' => <integer>,
        'TriggerName' => '<string>',
        'WorkerType' => 'Standard|G.1X|G.2X',
    ],
]

Result Details

Members

JobRun

Type: JobRun structure

The requested job-run metadata.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetJobRuns

$result = $client->getJobRuns([/* ... */]);
$promise = $client->getJobRunsAsync([/* ... */]);

Retrieves metadata for all runs of a given job definition.

Parameter Syntax

$result = $client->getJobRuns([
    'JobName' => '<string>', // REQUIRED
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

The name of the job definition for which to retrieve all job runs.

MaxResults

Type: int

The maximum size of the response.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'JobRuns' => [
        [
            'AllocatedCapacity' => <integer>,
            'Arguments' => ['<string>', ...],
            'Attempt' => <integer>,
            'CompletedOn' => <DateTime>,
            'ErrorMessage' => '<string>',
            'ExecutionTime' => <integer>,
            'Id' => '<string>',
            'JobName' => '<string>',
            'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
            'LastModifiedOn' => <DateTime>,
            'LogGroupName' => '<string>',
            'MaxCapacity' => <float>,
            'NotificationProperty' => [
                'NotifyDelayAfter' => <integer>,
            ],
            'NumberOfWorkers' => <integer>,
            'PredecessorRuns' => [
                [
                    'JobName' => '<string>',
                    'RunId' => '<string>',
                ],
                // ...
            ],
            'PreviousRunId' => '<string>',
            'SecurityConfiguration' => '<string>',
            'StartedOn' => <DateTime>,
            'Timeout' => <integer>,
            'TriggerName' => '<string>',
            'WorkerType' => 'Standard|G.1X|G.2X',
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

JobRuns

Type: Array of JobRun structures

A list of job-run metatdata objects.

NextToken

Type: string

A continuation token, if not all reequested job runs have been returned.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetJobs

$result = $client->getJobs([/* ... */]);
$promise = $client->getJobsAsync([/* ... */]);

Retrieves all current job definitions.

Parameter Syntax

$result = $client->getJobs([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

MaxResults

Type: int

The maximum size of the response.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'Jobs' => [
        [
            'AllocatedCapacity' => <integer>,
            'Command' => [
                'Name' => '<string>',
                'ScriptLocation' => '<string>',
            ],
            'Connections' => [
                'Connections' => ['<string>', ...],
            ],
            'CreatedOn' => <DateTime>,
            'DefaultArguments' => ['<string>', ...],
            'Description' => '<string>',
            'ExecutionProperty' => [
                'MaxConcurrentRuns' => <integer>,
            ],
            'LastModifiedOn' => <DateTime>,
            'LogUri' => '<string>',
            'MaxCapacity' => <float>,
            'MaxRetries' => <integer>,
            'Name' => '<string>',
            'NotificationProperty' => [
                'NotifyDelayAfter' => <integer>,
            ],
            'NumberOfWorkers' => <integer>,
            'Role' => '<string>',
            'SecurityConfiguration' => '<string>',
            'Timeout' => <integer>,
            'WorkerType' => 'Standard|G.1X|G.2X',
        ],
        // ...
    ],
    'NextToken' => '<string>',
]

Result Details

Members

Jobs

Type: Array of Job structures

A list of job definitions.

NextToken

Type: string

A continuation token, if not all job definitions have yet been returned.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetMapping

$result = $client->getMapping([/* ... */]);
$promise = $client->getMappingAsync([/* ... */]);

Creates mappings.

Parameter Syntax

$result = $client->getMapping([
    'Location' => [
        'DynamoDB' => [
            [
                'Name' => '<string>', // REQUIRED
                'Param' => true || false,
                'Value' => '<string>', // REQUIRED
            ],
            // ...
        ],
        'Jdbc' => [
            [
                'Name' => '<string>', // REQUIRED
                'Param' => true || false,
                'Value' => '<string>', // REQUIRED
            ],
            // ...
        ],
        'S3' => [
            [
                'Name' => '<string>', // REQUIRED
                'Param' => true || false,
                'Value' => '<string>', // REQUIRED
            ],
            // ...
        ],
    ],
    'Sinks' => [
        [
            'DatabaseName' => '<string>', // REQUIRED
            'TableName' => '<string>', // REQUIRED
        ],
        // ...
    ],
    'Source' => [ // REQUIRED
        'DatabaseName' => '<string>', // REQUIRED
        'TableName' => '<string>', // REQUIRED
    ],
]);

Parameter Details

Members

Location

Type: Location structure

Parameters for the mapping.

Sinks

Type: Array of CatalogEntry structures

A list of target tables.

Source

Required: Yes
Type: CatalogEntry structure

Specifies the source table.

Result Syntax

[
    'Mapping' => [
        [
            'SourcePath' => '<string>',
            'SourceTable' => '<string>',
            'SourceType' => '<string>',
            'TargetPath' => '<string>',
            'TargetTable' => '<string>',
            'TargetType' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

Mapping

Required: Yes
Type: Array of MappingEntry structures

A list of mappings to the specified targets.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
EntityNotFoundException:
A specified entity does not exist

GetPartition

$result = $client->getPartition([/* ... */]);
$promise = $client->getPartitionAsync([/* ... */]);

Retrieves information about a specified partition.

Parameter Syntax

$result = $client->getPartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionValues' => ['<string>', ...], // REQUIRED
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the partition in question resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the partition resides.

PartitionValues

Required: Yes
Type: Array of strings

The values that define the partition.

TableName

Required: Yes
Type: string

The name of the partition's table.

Result Syntax

[
    'Partition' => [
        'CreationTime' => <DateTime>,
        'DatabaseName' => '<string>',
        'LastAccessTime' => <DateTime>,
        'LastAnalyzedTime' => <DateTime>,
        'Parameters' => ['<string>', ...],
        'StorageDescriptor' => [
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>',
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>',
                    'SortOrder' => <integer>,
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'TableName' => '<string>',
        'Values' => ['<string>', ...],
    ],
]

Result Details

Members

Partition

Type: Partition structure

The requested information, in the form of a Partition object.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetPartitions

$result = $client->getPartitions([/* ... */]);
$promise = $client->getPartitionsAsync([/* ... */]);

Retrieves information about the partitions in a table.

Parameter Syntax

$result = $client->getPartitions([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'Expression' => '<string>',
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'Segment' => [
        'SegmentNumber' => <integer>, // REQUIRED
        'TotalSegments' => <integer>, // REQUIRED
    ],
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the partitions reside.

Expression

Type: string

An expression filtering the partitions to be returned.

The expression uses SQL syntax similar to the SQL WHERE filter clause. The SQL statement parser JSQLParser parses the expression.

Operators: The following are the operators that you can use in the Expression API call:

=

Checks if the values of the two operands are equal or not; if yes, then the condition becomes true.

Example: Assume 'variable a' holds 10 and 'variable b' holds 20.

(a = b) is not true.

< >

Checks if the values of two operands are equal or not; if the values are not equal, then the condition becomes true.

Example: (a < > b) is true.

>

Checks if the value of the left operand is greater than the value of the right operand; if yes, then the condition becomes true.

Example: (a > b) is not true.

<

Checks if the value of the left operand is less than the value of the right operand; if yes, then the condition becomes true.

Example: (a < b) is true.

>=

Checks if the value of the left operand is greater than or equal to the value of the right operand; if yes, then the condition becomes true.

Example: (a >= b) is not true.

<=

Checks if the value of the left operand is less than or equal to the value of the right operand; if yes, then the condition becomes true.

Example: (a <= b) is true.

AND, OR, IN, BETWEEN, LIKE, NOT, IS NULL

Logical operators.

Supported Partition Key Types: The following are the the supported partition keys.

string
date
timestamp
int
bigint
long
tinyint
smallint
decimal

If an invalid type is encountered, an exception is thrown.

The following list shows the valid operators on each type. When you define a crawler, the partitionKey type is created as a STRING, to be compatible with the catalog partitions.

Sample API Call:

MaxResults

Type: int

The maximum number of partitions to return in a single response.

NextToken

Type: string

A continuation token, if this is not the first call to retrieve these partitions.

Segment

Type: Segment structure

The segment of the table's partitions to scan in this request.

TableName

Required: Yes
Type: string

The name of the partitions' table.

Result Syntax

[
    'NextToken' => '<string>',
    'Partitions' => [
        [
            'CreationTime' => <DateTime>,
            'DatabaseName' => '<string>',
            'LastAccessTime' => <DateTime>,
            'LastAnalyzedTime' => <DateTime>,
            'Parameters' => ['<string>', ...],
            'StorageDescriptor' => [
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>',
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>',
                        'SortOrder' => <integer>,
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'TableName' => '<string>',
            'Values' => ['<string>', ...],
        ],
        // ...
    ],
]

Result Details

Members

NextToken

Type: string

A continuation token, if the returned list of partitions does not does not include the last one.

Partitions

Type: Array of Partition structures

A list of requested partitions.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.
InternalServiceException:
An internal service error occurred.
GlueEncryptionException:
An encryption operation failed.

GetPlan

$result = $client->getPlan([/* ... */]);
$promise = $client->getPlanAsync([/* ... */]);

Gets code to perform a specified mapping.

Parameter Syntax

$result = $client->getPlan([
    'Language' => 'PYTHON|SCALA',
    'Location' => [
        'DynamoDB' => [
            [
                'Name' => '<string>', // REQUIRED
                'Param' => true || false,
                'Value' => '<string>', // REQUIRED
            ],
            // ...
        ],
        'Jdbc' => [
            [
                'Name' => '<string>', // REQUIRED
                'Param' => true || false,
                'Value' => '<string>', // REQUIRED
            ],
            // ...
        ],
        'S3' => [
            [
                'Name' => '<string>', // REQUIRED
                'Param' => true || false,
                'Value' => '<string>', // REQUIRED
            ],
            // ...
        ],
    ],
    'Mapping' => [ // REQUIRED
        [
            'SourcePath' => '<string>',
            'SourceTable' => '<string>',
            'SourceType' => '<string>',
            'TargetPath' => '<string>',
            'TargetTable' => '<string>',
            'TargetType' => '<string>',
        ],
        // ...
    ],
    'Sinks' => [
        [
            'DatabaseName' => '<string>', // REQUIRED
            'TableName' => '<string>', // REQUIRED
        ],
        // ...
    ],
    'Source' => [ // REQUIRED
        'DatabaseName' => '<string>', // REQUIRED
        'TableName' => '<string>', // REQUIRED
    ],
]);

Parameter Details

Members

Language

Type: string

The programming language of the code to perform the mapping.

Location

Type: Location structure

Parameters for the mapping.

Mapping

Required: Yes
Type: Array of MappingEntry structures

The list of mappings from a source table to target tables.

Sinks

Type: Array of CatalogEntry structures

The target tables.

Source

Required: Yes
Type: CatalogEntry structure

The source table.

Result Syntax

[
    'PythonScript' => '<string>',
    'ScalaCode' => '<string>',
]

Result Details

Members

PythonScript

Type: string

A Python script to perform the mapping.

ScalaCode

Type: string

Scala code to perform the mapping.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetResourcePolicy

$result = $client->getResourcePolicy([/* ... */]);
$promise = $client->getResourcePolicyAsync([/* ... */]);

Retrieves a specified resource policy.

Parameter Syntax

$result = $client->getResourcePolicy([
]);

Parameter Details

Members

Result Syntax

[
    'CreateTime' => <DateTime>,
    'PolicyHash' => '<string>',
    'PolicyInJson' => '<string>',
    'UpdateTime' => <DateTime>,
]

Result Details

Members

CreateTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The date and time at which the policy was created.

PolicyHash

Type: string

Contains the hash value associated with this policy.

PolicyInJson

Type: string

Contains the requested policy document, in JSON format.

UpdateTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The date and time at which the policy was last updated.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.

GetSecurityConfiguration

$result = $client->getSecurityConfiguration([/* ... */]);
$promise = $client->getSecurityConfigurationAsync([/* ... */]);

Retrieves a specified security configuration.

Parameter Syntax

$result = $client->getSecurityConfiguration([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the security configuration to retrieve.

Result Syntax

[
    'SecurityConfiguration' => [
        'CreatedTimeStamp' => <DateTime>,
        'EncryptionConfiguration' => [
            'CloudWatchEncryption' => [
                'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
                'KmsKeyArn' => '<string>',
            ],
            'JobBookmarksEncryption' => [
                'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
                'KmsKeyArn' => '<string>',
            ],
            'S3Encryption' => [
                [
                    'KmsKeyArn' => '<string>',
                    'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
                ],
                // ...
            ],
        ],
        'Name' => '<string>',
    ],
]

Result Details

Members

SecurityConfiguration

Type: SecurityConfiguration structure

The requested security configuration

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetSecurityConfigurations

$result = $client->getSecurityConfigurations([/* ... */]);
$promise = $client->getSecurityConfigurationsAsync([/* ... */]);

Retrieves a list of all security configurations.

Parameter Syntax

$result = $client->getSecurityConfigurations([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

MaxResults

Type: int

The maximum number of results to return.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'NextToken' => '<string>',
    'SecurityConfigurations' => [
        [
            'CreatedTimeStamp' => <DateTime>,
            'EncryptionConfiguration' => [
                'CloudWatchEncryption' => [
                    'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
                    'KmsKeyArn' => '<string>',
                ],
                'JobBookmarksEncryption' => [
                    'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
                    'KmsKeyArn' => '<string>',
                ],
                'S3Encryption' => [
                    [
                        'KmsKeyArn' => '<string>',
                        'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
                    ],
                    // ...
                ],
            ],
            'Name' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

NextToken

Type: string

A continuation token, if there are more security configurations to return.

SecurityConfigurations

Type: Array of SecurityConfiguration structures

A list of security configurations.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetTable

$result = $client->getTable([/* ... */]);
$promise = $client->getTableAsync([/* ... */]);

Retrieves the Table definition in a Data Catalog for a specified table.

Parameter Syntax

$result = $client->getTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.

Name

Required: Yes
Type: string

The name of the table for which to retrieve the definition. For Hive compatibility, this name is entirely lowercase.

Result Syntax

[
    'Table' => [
        'CreateTime' => <DateTime>,
        'CreatedBy' => '<string>',
        'DatabaseName' => '<string>',
        'Description' => '<string>',
        'LastAccessTime' => <DateTime>,
        'LastAnalyzedTime' => <DateTime>,
        'Name' => '<string>',
        'Owner' => '<string>',
        'Parameters' => ['<string>', ...],
        'PartitionKeys' => [
            [
                'Comment' => '<string>',
                'Name' => '<string>',
                'Type' => '<string>',
            ],
            // ...
        ],
        'Retention' => <integer>,
        'StorageDescriptor' => [
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>',
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>',
                    'SortOrder' => <integer>,
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'TableType' => '<string>',
        'UpdateTime' => <DateTime>,
        'ViewExpandedText' => '<string>',
        'ViewOriginalText' => '<string>',
    ],
]

Result Details

Members

Table

Type: Table structure

The Table object that defines the specified table.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetTableVersion

$result = $client->getTableVersion([/* ... */]);
$promise = $client->getTableVersionAsync([/* ... */]);

Retrieves a specified version of a table.

Parameter Syntax

$result = $client->getTableVersion([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'TableName' => '<string>', // REQUIRED
    'VersionId' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.

TableName

Required: Yes
Type: string

The name of the table. For Hive compatibility, this name is entirely lowercase.

VersionId

Type: string

The ID value of the table version to be retrieved. A VersionID is a string representation of an integer. Each version is incremented by 1.

Result Syntax

[
    'TableVersion' => [
        'Table' => [
            'CreateTime' => <DateTime>,
            'CreatedBy' => '<string>',
            'DatabaseName' => '<string>',
            'Description' => '<string>',
            'LastAccessTime' => <DateTime>,
            'LastAnalyzedTime' => <DateTime>,
            'Name' => '<string>',
            'Owner' => '<string>',
            'Parameters' => ['<string>', ...],
            'PartitionKeys' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>',
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Retention' => <integer>,
            'StorageDescriptor' => [
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>',
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>',
                        'SortOrder' => <integer>,
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'TableType' => '<string>',
            'UpdateTime' => <DateTime>,
            'ViewExpandedText' => '<string>',
            'ViewOriginalText' => '<string>',
        ],
        'VersionId' => '<string>',
    ],
]

Result Details

Members

TableVersion

Type: TableVersion structure

The requested table version.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetTableVersions

$result = $client->getTableVersions([/* ... */]);
$promise = $client->getTableVersionsAsync([/* ... */]);

Retrieves a list of strings that identify available versions of a specified table.

Parameter Syntax

$result = $client->getTableVersions([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.

MaxResults

Type: int

The maximum number of table versions to return in one response.

NextToken

Type: string

A continuation token, if this is not the first call.

TableName

Required: Yes
Type: string

The name of the table. For Hive compatibility, this name is entirely lowercase.

Result Syntax

[
    'NextToken' => '<string>',
    'TableVersions' => [
        [
            'Table' => [
                'CreateTime' => <DateTime>,
                'CreatedBy' => '<string>',
                'DatabaseName' => '<string>',
                'Description' => '<string>',
                'LastAccessTime' => <DateTime>,
                'LastAnalyzedTime' => <DateTime>,
                'Name' => '<string>',
                'Owner' => '<string>',
                'Parameters' => ['<string>', ...],
                'PartitionKeys' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>',
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Retention' => <integer>,
                'StorageDescriptor' => [
                    'BucketColumns' => ['<string>', ...],
                    'Columns' => [
                        [
                            'Comment' => '<string>',
                            'Name' => '<string>',
                            'Type' => '<string>',
                        ],
                        // ...
                    ],
                    'Compressed' => true || false,
                    'InputFormat' => '<string>',
                    'Location' => '<string>',
                    'NumberOfBuckets' => <integer>,
                    'OutputFormat' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerdeInfo' => [
                        'Name' => '<string>',
                        'Parameters' => ['<string>', ...],
                        'SerializationLibrary' => '<string>',
                    ],
                    'SkewedInfo' => [
                        'SkewedColumnNames' => ['<string>', ...],
                        'SkewedColumnValueLocationMaps' => ['<string>', ...],
                        'SkewedColumnValues' => ['<string>', ...],
                    ],
                    'SortColumns' => [
                        [
                            'Column' => '<string>',
                            'SortOrder' => <integer>,
                        ],
                        // ...
                    ],
                    'StoredAsSubDirectories' => true || false,
                ],
                'TableType' => '<string>',
                'UpdateTime' => <DateTime>,
                'ViewExpandedText' => '<string>',
                'ViewOriginalText' => '<string>',
            ],
            'VersionId' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

NextToken

Type: string

A continuation token, if the list of available versions does not include the last one.

TableVersions

Type: Array of TableVersion structures

A list of strings identifying available versions of the specified table.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetTables

$result = $client->getTables([/* ... */]);
$promise = $client->getTablesAsync([/* ... */]);

Retrieves the definitions of some or all of the tables in a given Database.

Parameter Syntax

$result = $client->getTables([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'Expression' => '<string>',
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The database in the catalog whose tables to list. For Hive compatibility, this name is entirely lowercase.

Expression

Type: string

A regular expression pattern. If present, only those tables whose names match the pattern are returned.

MaxResults

Type: int

The maximum number of tables to return in a single response.

NextToken

Type: string

A continuation token, included if this is a continuation call.

Result Syntax

[
    'NextToken' => '<string>',
    'TableList' => [
        [
            'CreateTime' => <DateTime>,
            'CreatedBy' => '<string>',
            'DatabaseName' => '<string>',
            'Description' => '<string>',
            'LastAccessTime' => <DateTime>,
            'LastAnalyzedTime' => <DateTime>,
            'Name' => '<string>',
            'Owner' => '<string>',
            'Parameters' => ['<string>', ...],
            'PartitionKeys' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>',
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Retention' => <integer>,
            'StorageDescriptor' => [
                'BucketColumns' => ['<string>', ...],
                'Columns' => [
                    [
                        'Comment' => '<string>',
                        'Name' => '<string>',
                        'Type' => '<string>',
                    ],
                    // ...
                ],
                'Compressed' => true || false,
                'InputFormat' => '<string>',
                'Location' => '<string>',
                'NumberOfBuckets' => <integer>,
                'OutputFormat' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerdeInfo' => [
                    'Name' => '<string>',
                    'Parameters' => ['<string>', ...],
                    'SerializationLibrary' => '<string>',
                ],
                'SkewedInfo' => [
                    'SkewedColumnNames' => ['<string>', ...],
                    'SkewedColumnValueLocationMaps' => ['<string>', ...],
                    'SkewedColumnValues' => ['<string>', ...],
                ],
                'SortColumns' => [
                    [
                        'Column' => '<string>',
                        'SortOrder' => <integer>,
                    ],
                    // ...
                ],
                'StoredAsSubDirectories' => true || false,
            ],
            'TableType' => '<string>',
            'UpdateTime' => <DateTime>,
            'ViewExpandedText' => '<string>',
            'ViewOriginalText' => '<string>',
        ],
        // ...
    ],
]

Result Details

Members

NextToken

Type: string

A continuation token, present if the current list segment is not the last.

TableList

Type: Array of Table structures

A list of the requested Table objects.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.
InternalServiceException:
An internal service error occurred.
GlueEncryptionException:
An encryption operation failed.

GetTags

$result = $client->getTags([/* ... */]);
$promise = $client->getTagsAsync([/* ... */]);

Retrieves a list of tags associated with a resource.

Parameter Syntax

$result = $client->getTags([
    'ResourceArn' => '<string>', // REQUIRED
]);

Parameter Details

Members

ResourceArn

Required: Yes
Type: string

The Amazon ARN of the resource for which to retrieve tags.

Result Syntax

[
    'Tags' => ['<string>', ...],
]

Result Details

Members

Tags

Type: Associative array of custom strings keys (TagKey) to strings

The requested tags.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
EntityNotFoundException:
A specified entity does not exist

GetTrigger

$result = $client->getTrigger([/* ... */]);
$promise = $client->getTriggerAsync([/* ... */]);

Retrieves the definition of a trigger.

Parameter Syntax

$result = $client->getTrigger([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the trigger to retrieve.

Result Syntax

[
    'Trigger' => [
        'Actions' => [
            [
                'Arguments' => ['<string>', ...],
                'JobName' => '<string>',
                'NotificationProperty' => [
                    'NotifyDelayAfter' => <integer>,
                ],
                'SecurityConfiguration' => '<string>',
                'Timeout' => <integer>,
            ],
            // ...
        ],
        'Description' => '<string>',
        'Id' => '<string>',
        'Name' => '<string>',
        'Predicate' => [
            'Conditions' => [
                [
                    'JobName' => '<string>',
                    'LogicalOperator' => 'EQUALS',
                    'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
                ],
                // ...
            ],
            'Logical' => 'AND|ANY',
        ],
        'Schedule' => '<string>',
        'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
        'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
    ],
]

Result Details

Members

Trigger

Type: Trigger structure

The requested trigger definition.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetTriggers

$result = $client->getTriggers([/* ... */]);
$promise = $client->getTriggersAsync([/* ... */]);

Gets all the triggers associated with a job.

Parameter Syntax

$result = $client->getTriggers([
    'DependentJobName' => '<string>',
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
]);

Parameter Details

Members

DependentJobName

Type: string

The name of the job for which to retrieve triggers. The trigger that can start this job will be returned, and if there is no such trigger, all triggers will be returned.

MaxResults

Type: int

The maximum size of the response.

NextToken

Type: string

A continuation token, if this is a continuation call.

Result Syntax

[
    'NextToken' => '<string>',
    'Triggers' => [
        [
            'Actions' => [
                [
                    'Arguments' => ['<string>', ...],
                    'JobName' => '<string>',
                    'NotificationProperty' => [
                        'NotifyDelayAfter' => <integer>,
                    ],
                    'SecurityConfiguration' => '<string>',
                    'Timeout' => <integer>,
                ],
                // ...
            ],
            'Description' => '<string>',
            'Id' => '<string>',
            'Name' => '<string>',
            'Predicate' => [
                'Conditions' => [
                    [
                        'JobName' => '<string>',
                        'LogicalOperator' => 'EQUALS',
                        'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
                    ],
                    // ...
                ],
                'Logical' => 'AND|ANY',
            ],
            'Schedule' => '<string>',
            'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
            'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
        ],
        // ...
    ],
]

Result Details

Members

NextToken

Type: string

A continuation token, if not all the requested triggers have yet been returned.

Triggers

Type: Array of Trigger structures

A list of triggers for the specified job.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

GetUserDefinedFunction

$result = $client->getUserDefinedFunction([/* ... */]);
$promise = $client->getUserDefinedFunctionAsync([/* ... */]);

Retrieves a specified function definition from the Data Catalog.

Parameter Syntax

$result = $client->getUserDefinedFunction([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'FunctionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the function to be retrieved is located. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the function is located.

FunctionName

Required: Yes
Type: string

The name of the function.

Result Syntax

[
    'UserDefinedFunction' => [
        'ClassName' => '<string>',
        'CreateTime' => <DateTime>,
        'FunctionName' => '<string>',
        'OwnerName' => '<string>',
        'OwnerType' => 'USER|ROLE|GROUP',
        'ResourceUris' => [
            [
                'ResourceType' => 'JAR|FILE|ARCHIVE',
                'Uri' => '<string>',
            ],
            // ...
        ],
    ],
]

Result Details

Members

UserDefinedFunction

Type: UserDefinedFunction structure

The requested function definition.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

GetUserDefinedFunctions

$result = $client->getUserDefinedFunctions([/* ... */]);
$promise = $client->getUserDefinedFunctionsAsync([/* ... */]);

Retrieves a multiple function definitions from the Data Catalog.

Parameter Syntax

$result = $client->getUserDefinedFunctions([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'Pattern' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the functions to be retrieved are located. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the functions are located.

MaxResults

Type: int

The maximum number of functions to return in one response.

NextToken

Type: string

A continuation token, if this is a continuation call.

Pattern

Required: Yes
Type: string

An optional function-name pattern string that filters the function definitions returned.

Result Syntax

[
    'NextToken' => '<string>',
    'UserDefinedFunctions' => [
        [
            'ClassName' => '<string>',
            'CreateTime' => <DateTime>,
            'FunctionName' => '<string>',
            'OwnerName' => '<string>',
            'OwnerType' => 'USER|ROLE|GROUP',
            'ResourceUris' => [
                [
                    'ResourceType' => 'JAR|FILE|ARCHIVE',
                    'Uri' => '<string>',
                ],
                // ...
            ],
        ],
        // ...
    ],
]

Result Details

Members

NextToken

Type: string

A continuation token, if the list of functions returned does not include the last requested function.

UserDefinedFunctions

Type: Array of UserDefinedFunction structures

A list of requested function definitions.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.
InternalServiceException:
An internal service error occurred.
GlueEncryptionException:
An encryption operation failed.

ImportCatalogToGlue

$result = $client->importCatalogToGlue([/* ... */]);
$promise = $client->importCatalogToGlueAsync([/* ... */]);

Imports an existing Athena Data Catalog to AWS Glue

Parameter Syntax

$result = $client->importCatalogToGlue([
    'CatalogId' => '<string>',
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the catalog to import. Currently, this should be the AWS account ID.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

ListCrawlers

$result = $client->listCrawlers([/* ... */]);
$promise = $client->listCrawlersAsync([/* ... */]);

Retrieves the names of all crawler resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.

This operation takes the optional Tags field which you can use as a filter on the response so that tagged resources can be retrieved as a group. If you choose to use tags filtering, only resources with the tag will be retrieved.

Parameter Syntax

$result = $client->listCrawlers([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'Tags' => ['<string>', ...],
]);

Parameter Details

Members

MaxResults

Type: int

The maximum size of a list to return.

NextToken

Type: string

A continuation token, if this is a continuation request.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

Specifies to return only these tagged resources.

Result Syntax

[
    'CrawlerNames' => ['<string>', ...],
    'NextToken' => '<string>',
]

Result Details

Members

CrawlerNames

Type: Array of strings

The names of all crawlers in the account, or the crawlers with the specified tags.

NextToken

Type: string

A continuation token, if the returned list does not contain the last metric available.

Errors

OperationTimeoutException:
The operation timed out.

ListDevEndpoints

$result = $client->listDevEndpoints([/* ... */]);
$promise = $client->listDevEndpointsAsync([/* ... */]);

Retrieves the names of all DevEndpoint resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.

Parameter Syntax

$result = $client->listDevEndpoints([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'Tags' => ['<string>', ...],
]);

Parameter Details

Members

MaxResults

Type: int

The maximum size of a list to return.

NextToken

Type: string

A continuation token, if this is a continuation request.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

Specifies to return only these tagged resources.

Result Syntax

[
    'DevEndpointNames' => ['<string>', ...],
    'NextToken' => '<string>',
]

Result Details

Members

DevEndpointNames

Type: Array of strings

The names of all DevEndpoints in the account, or the DevEndpoints with the specified tags.

NextToken

Type: string

A continuation token, if the returned list does not contain the last metric available.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

ListJobs

$result = $client->listJobs([/* ... */]);
$promise = $client->listJobsAsync([/* ... */]);

Retrieves the names of all job resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.

Parameter Syntax

$result = $client->listJobs([
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'Tags' => ['<string>', ...],
]);

Parameter Details

Members

MaxResults

Type: int

The maximum size of a list to return.

NextToken

Type: string

A continuation token, if this is a continuation request.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

Specifies to return only these tagged resources.

Result Syntax

[
    'JobNames' => ['<string>', ...],
    'NextToken' => '<string>',
]

Result Details

Members

JobNames

Type: Array of strings

The names of all jobs in the account, or the jobs with the specified tags.

NextToken

Type: string

A continuation token, if the returned list does not contain the last metric available.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

ListTriggers

$result = $client->listTriggers([/* ... */]);
$promise = $client->listTriggersAsync([/* ... */]);

Retrieves the names of all trigger resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.

Parameter Syntax

$result = $client->listTriggers([
    'DependentJobName' => '<string>',
    'MaxResults' => <integer>,
    'NextToken' => '<string>',
    'Tags' => ['<string>', ...],
]);

Parameter Details

Members

DependentJobName

Type: string

The name of the job for which to retrieve triggers. The trigger that can start this job will be returned, and if there is no such trigger, all triggers will be returned.

MaxResults

Type: int

The maximum size of a list to return.

NextToken

Type: string

A continuation token, if this is a continuation request.

Tags

Type: Associative array of custom strings keys (TagKey) to strings

Specifies to return only these tagged resources.

Result Syntax

[
    'NextToken' => '<string>',
    'TriggerNames' => ['<string>', ...],
]

Result Details

Members

NextToken

Type: string

A continuation token, if the returned list does not contain the last metric available.

TriggerNames

Type: Array of strings

The names of all triggers in the account, or the triggers with the specified tags.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

PutDataCatalogEncryptionSettings

$result = $client->putDataCatalogEncryptionSettings([/* ... */]);
$promise = $client->putDataCatalogEncryptionSettingsAsync([/* ... */]);

Sets the security configuration for a specified catalog. After the configuration has been set, the specified encryption is applied to every catalog write thereafter.

Parameter Syntax

$result = $client->putDataCatalogEncryptionSettings([
    'CatalogId' => '<string>',
    'DataCatalogEncryptionSettings' => [ // REQUIRED
        'ConnectionPasswordEncryption' => [
            'AwsKmsKeyId' => '<string>',
            'ReturnConnectionPasswordEncrypted' => true || false, // REQUIRED
        ],
        'EncryptionAtRest' => [
            'CatalogEncryptionMode' => 'DISABLED|SSE-KMS', // REQUIRED
            'SseAwsKmsKeyId' => '<string>',
        ],
    ],
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog for which to set the security configuration. If none is provided, the AWS account ID is used by default.

DataCatalogEncryptionSettings

Required: Yes
Type: DataCatalogEncryptionSettings structure

The security configuration to set.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InternalServiceException:
An internal service error occurred.
InvalidInputException:
The input provided was not valid.
OperationTimeoutException:
The operation timed out.

PutResourcePolicy

$result = $client->putResourcePolicy([/* ... */]);
$promise = $client->putResourcePolicyAsync([/* ... */]);

Sets the Data Catalog resource policy for access control.

Parameter Syntax

$result = $client->putResourcePolicy([
    'PolicyExistsCondition' => 'MUST_EXIST|NOT_EXIST|NONE',
    'PolicyHashCondition' => '<string>',
    'PolicyInJson' => '<string>', // REQUIRED
]);

Parameter Details

Members

PolicyExistsCondition

Type: string

A value of MUST_EXIST is used to update a policy. A value of NOT_EXIST is used to create a new policy. If a value of NONE or a null value is used, the call will not depend on the existence of a policy.

PolicyHashCondition

Type: string

The hash value returned when the previous policy was set using PutResourcePolicy. Its purpose is to prevent concurrent modifications of a policy. Do not use this parameter if no previous policy has been set.

PolicyInJson

Required: Yes
Type: string

Contains the policy document to set, in JSON format.

Result Syntax

[
    'PolicyHash' => '<string>',
]

Result Details

Members

PolicyHash

Type: string

A hash of the policy that has just been set. This must be included in a subsequent call that overwrites or updates this policy.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
ConditionCheckFailureException:
A specified condition was not satisfied.

ResetJobBookmark

$result = $client->resetJobBookmark([/* ... */]);
$promise = $client->resetJobBookmarkAsync([/* ... */]);

Resets a bookmark entry.

Parameter Syntax

$result = $client->resetJobBookmark([
    'JobName' => '<string>', // REQUIRED
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

The name of the job in question.

Result Syntax

[
    'JobBookmarkEntry' => [
        'Attempt' => <integer>,
        'JobBookmark' => '<string>',
        'JobName' => '<string>',
        'Run' => <integer>,
        'Version' => <integer>,
    ],
]

Result Details

Members

JobBookmarkEntry

Type: JobBookmarkEntry structure

The reset bookmark entry.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.

StartCrawler

$result = $client->startCrawler([/* ... */]);
$promise = $client->startCrawlerAsync([/* ... */]);

Starts a crawl using the specified crawler, regardless of what is scheduled. If the crawler is already running, returns a CrawlerRunningException.

Parameter Syntax

$result = $client->startCrawler([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

Name of the crawler to start.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
CrawlerRunningException:
The operation cannot be performed because the crawler is already running.
OperationTimeoutException:
The operation timed out.

StartCrawlerSchedule

$result = $client->startCrawlerSchedule([/* ... */]);
$promise = $client->startCrawlerScheduleAsync([/* ... */]);

Changes the schedule state of the specified crawler to SCHEDULED, unless the crawler is already running or the schedule state is already SCHEDULED.

Parameter Syntax

$result = $client->startCrawlerSchedule([
    'CrawlerName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CrawlerName

Required: Yes
Type: string

Name of the crawler to schedule.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
SchedulerRunningException:
The specified scheduler is already running.
SchedulerTransitioningException:
The specified scheduler is transitioning.
NoScheduleException:
There is no applicable schedule.
OperationTimeoutException:
The operation timed out.

StartJobRun

$result = $client->startJobRun([/* ... */]);
$promise = $client->startJobRunAsync([/* ... */]);

Starts a job run using a job definition.

Parameter Syntax

$result = $client->startJobRun([
    'AllocatedCapacity' => <integer>,
    'Arguments' => ['<string>', ...],
    'JobName' => '<string>', // REQUIRED
    'JobRunId' => '<string>',
    'MaxCapacity' => <float>,
    'NotificationProperty' => [
        'NotifyDelayAfter' => <integer>,
    ],
    'NumberOfWorkers' => <integer>,
    'SecurityConfiguration' => '<string>',
    'Timeout' => <integer>,
    'WorkerType' => 'Standard|G.1X|G.2X',
]);

Parameter Details

Members

AllocatedCapacity

Type: int

This field is deprecated, use MaxCapacity instead.

The number of AWS Glue data processing units (DPUs) to allocate to this JobRun. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.

Arguments

Type: Associative array of custom strings keys (GenericString) to strings

The job arguments specifically for this run. For this job run, they replace the default arguments set in the job definition itself.

You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.

For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.

For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

JobName

Required: Yes
Type: string

The name of the job definition to use.

JobRunId

Type: string

The ID of a previous JobRun to retry.

MaxCapacity

Type: double

Do not set Max Capacity if using WorkerType and NumberOfWorkers.

The value that can be allocated for MaxCapacity depends on whether you are running a python shell job, or an Apache Spark ETL job:

When you specify a python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
When you specify an Apache Spark ETL job (JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

NotificationProperty

Type: NotificationProperty structure

Specifies configuration properties of a job run notification.

NumberOfWorkers

Type: int

The number of workers of a defined workerType that are allocated when a job runs.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this job run.

Timeout

Type: int

The JobRun timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours). This overrides the timeout value set in the parent job.

WorkerType

Type: string

The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

Result Syntax

[
    'JobRunId' => '<string>',
]

Result Details

Members

JobRunId

Type: string

The ID assigned to this job run.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
ConcurrentRunsExceededException:
Too many jobs are being run concurrently.

StartTrigger

$result = $client->startTrigger([/* ... */]);
$promise = $client->startTriggerAsync([/* ... */]);

Starts an existing trigger. See Triggering Jobs for information about how different types of trigger are started.

Parameter Syntax

$result = $client->startTrigger([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the trigger to start.

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Members

Name

Type: string

The name of the trigger that was started.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
ConcurrentRunsExceededException:
Too many jobs are being run concurrently.

StopCrawler

$result = $client->stopCrawler([/* ... */]);
$promise = $client->stopCrawlerAsync([/* ... */]);

If the specified crawler is running, stops the crawl.

Parameter Syntax

$result = $client->stopCrawler([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

Name of the crawler to stop.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
CrawlerNotRunningException:
The specified crawler is not running.
CrawlerStoppingException:
The specified crawler is stopping.
OperationTimeoutException:
The operation timed out.

StopCrawlerSchedule

$result = $client->stopCrawlerSchedule([/* ... */]);
$promise = $client->stopCrawlerScheduleAsync([/* ... */]);

Sets the schedule state of the specified crawler to NOT_SCHEDULED, but does not stop the crawler if it is already running.

Parameter Syntax

$result = $client->stopCrawlerSchedule([
    'CrawlerName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CrawlerName

Required: Yes
Type: string

Name of the crawler whose schedule state to set.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
SchedulerNotRunningException:
The specified scheduler is not running.
SchedulerTransitioningException:
The specified scheduler is transitioning.
OperationTimeoutException:
The operation timed out.

StopTrigger

$result = $client->stopTrigger([/* ... */]);
$promise = $client->stopTriggerAsync([/* ... */]);

Stops a specified trigger.

Parameter Syntax

$result = $client->stopTrigger([
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the trigger to stop.

Result Syntax

[
    'Name' => '<string>',
]

Result Details

Members

Name

Type: string

The name of the trigger that was stopped.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.

TagResource

$result = $client->tagResource([/* ... */]);
$promise = $client->tagResourceAsync([/* ... */]);

Adds tags to a resource. A tag is a label you can assign to an AWS resource. In AWS Glue, you can tag only certain resources. For information about what resources you can tag, see AWS Tags in AWS Glue.

Parameter Syntax

$result = $client->tagResource([
    'ResourceArn' => '<string>', // REQUIRED
    'TagsToAdd' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

ResourceArn

Required: Yes
Type: string

The ARN of the AWS Glue resource to which to add the tags. For more information about AWS Glue resource ARNs, see the AWS Glue ARN string pattern.

TagsToAdd

Required: Yes
Type: Associative array of custom strings keys (TagKey) to strings

Tags to add to this resource.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
EntityNotFoundException:
A specified entity does not exist

UntagResource

$result = $client->untagResource([/* ... */]);
$promise = $client->untagResourceAsync([/* ... */]);

Removes tags from a resource.

Parameter Syntax

$result = $client->untagResource([
    'ResourceArn' => '<string>', // REQUIRED
    'TagsToRemove' => ['<string>', ...], // REQUIRED
]);

Parameter Details

Members

ResourceArn

Required: Yes
Type: string

The ARN of the resource from which to remove the tags.

TagsToRemove

Required: Yes
Type: Array of strings

Tags to remove from this resource.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
EntityNotFoundException:
A specified entity does not exist

UpdateClassifier

$result = $client->updateClassifier([/* ... */]);
$promise = $client->updateClassifierAsync([/* ... */]);

Modifies an existing classifier (a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field is present).

Parameter Syntax

$result = $client->updateClassifier([
    'CsvClassifier' => [
        'AllowSingleColumn' => true || false,
        'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
        'Delimiter' => '<string>',
        'DisableValueTrimming' => true || false,
        'Header' => ['<string>', ...],
        'Name' => '<string>', // REQUIRED
        'QuoteSymbol' => '<string>',
    ],
    'GrokClassifier' => [
        'Classification' => '<string>',
        'CustomPatterns' => '<string>',
        'GrokPattern' => '<string>',
        'Name' => '<string>', // REQUIRED
    ],
    'JsonClassifier' => [
        'JsonPath' => '<string>',
        'Name' => '<string>', // REQUIRED
    ],
    'XMLClassifier' => [
        'Classification' => '<string>',
        'Name' => '<string>', // REQUIRED
        'RowTag' => '<string>',
    ],
]);

Parameter Details

Members

CsvClassifier

Type: UpdateCsvClassifierRequest structure

A CsvClassifier object with updated fields.

GrokClassifier

Type: UpdateGrokClassifierRequest structure

A GrokClassifier object with updated fields.

JsonClassifier

Type: UpdateJsonClassifierRequest structure

A JsonClassifier object with updated fields.

XMLClassifier

Type: UpdateXMLClassifierRequest structure

An XMLClassifier object with updated fields.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
VersionMismatchException:
There was a version conflict.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.

UpdateConnection

$result = $client->updateConnection([/* ... */]);
$promise = $client->updateConnectionAsync([/* ... */]);

Updates a connection definition in the Data Catalog.

Parameter Syntax

$result = $client->updateConnection([
    'CatalogId' => '<string>',
    'ConnectionInput' => [ // REQUIRED
        'ConnectionProperties' => ['<string>', ...], // REQUIRED
        'ConnectionType' => 'JDBC|SFTP', // REQUIRED
        'Description' => '<string>',
        'MatchCriteria' => ['<string>', ...],
        'Name' => '<string>', // REQUIRED
        'PhysicalConnectionRequirements' => [
            'AvailabilityZone' => '<string>',
            'SecurityGroupIdList' => ['<string>', ...],
            'SubnetId' => '<string>',
        ],
    ],
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default.

ConnectionInput

Required: Yes
Type: ConnectionInput structure

A ConnectionInput object that redefines the connection in question.

Name

Required: Yes
Type: string

The name of the connection definition to update.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
GlueEncryptionException:
An encryption operation failed.

UpdateCrawler

$result = $client->updateCrawler([/* ... */]);
$promise = $client->updateCrawlerAsync([/* ... */]);

Updates a crawler. If a crawler is running, you must stop it using StopCrawler before updating it.

Parameter Syntax

$result = $client->updateCrawler([
    'Classifiers' => ['<string>', ...],
    'Configuration' => '<string>',
    'CrawlerSecurityConfiguration' => '<string>',
    'DatabaseName' => '<string>',
    'Description' => '<string>',
    'Name' => '<string>', // REQUIRED
    'Role' => '<string>',
    'Schedule' => '<string>',
    'SchemaChangePolicy' => [
        'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
        'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
    ],
    'TablePrefix' => '<string>',
    'Targets' => [
        'DynamoDBTargets' => [
            [
                'Path' => '<string>',
            ],
            // ...
        ],
        'JdbcTargets' => [
            [
                'ConnectionName' => '<string>',
                'Exclusions' => ['<string>', ...],
                'Path' => '<string>',
            ],
            // ...
        ],
        'S3Targets' => [
            [
                'Exclusions' => ['<string>', ...],
                'Path' => '<string>',
            ],
            // ...
        ],
    ],
]);

Parameter Details

Members

Classifiers

Type: Array of strings

Configuration

Type: string

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

CrawlerSecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used by this Crawler.

DatabaseName

Type: string

The AWS Glue database where results are stored, such as: arn:aws:daylight:us-east-1::database/sometable/*.

Description

Type: string

A description of the new crawler.

Name

Required: Yes
Type: string

Name of the new crawler.

Role

Type: string

The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

Schedule

Type: string

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

SchemaChangePolicy

Type: SchemaChangePolicy structure

Policy for the crawler's update and deletion behavior.

TablePrefix

Type: string

The table prefix used for catalog tables that are created.

Targets

Type: CrawlerTargets structure

A list of targets to crawl.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

InvalidInputException:
The input provided was not valid.
VersionMismatchException:
There was a version conflict.
EntityNotFoundException:
A specified entity does not exist
CrawlerRunningException:
The operation cannot be performed because the crawler is already running.
OperationTimeoutException:
The operation timed out.

UpdateCrawlerSchedule

$result = $client->updateCrawlerSchedule([/* ... */]);
$promise = $client->updateCrawlerScheduleAsync([/* ... */]);

Updates the schedule of a crawler using a cron expression.

Parameter Syntax

$result = $client->updateCrawlerSchedule([
    'CrawlerName' => '<string>', // REQUIRED
    'Schedule' => '<string>',
]);

Parameter Details

Members

CrawlerName

Required: Yes
Type: string

Name of the crawler whose schedule to update.

Schedule

Type: string

The updated cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
VersionMismatchException:
There was a version conflict.
SchedulerTransitioningException:
The specified scheduler is transitioning.
OperationTimeoutException:
The operation timed out.

UpdateDatabase

$result = $client->updateDatabase([/* ... */]);
$promise = $client->updateDatabaseAsync([/* ... */]);

Updates an existing database definition in a Data Catalog.

Parameter Syntax

$result = $client->updateDatabase([
    'CatalogId' => '<string>',
    'DatabaseInput' => [ // REQUIRED
        'Description' => '<string>',
        'LocationUri' => '<string>',
        'Name' => '<string>', // REQUIRED
        'Parameters' => ['<string>', ...],
    ],
    'Name' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog in which the metadata database resides. If none is supplied, the AWS account ID is used by default.

DatabaseInput

Required: Yes
Type: DatabaseInput structure

A DatabaseInput object specifying the new definition of the metadata database in the catalog.

Name

Required: Yes
Type: string

The name of the database to update in the catalog. For Hive compatibility, this is folded to lowercase.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

UpdateDevEndpoint

$result = $client->updateDevEndpoint([/* ... */]);
$promise = $client->updateDevEndpointAsync([/* ... */]);

Updates a specified DevEndpoint.

Parameter Syntax

$result = $client->updateDevEndpoint([
    'AddArguments' => ['<string>', ...],
    'AddPublicKeys' => ['<string>', ...],
    'CustomLibraries' => [
        'ExtraJarsS3Path' => '<string>',
        'ExtraPythonLibsS3Path' => '<string>',
    ],
    'DeleteArguments' => ['<string>', ...],
    'DeletePublicKeys' => ['<string>', ...],
    'EndpointName' => '<string>', // REQUIRED
    'PublicKey' => '<string>',
    'UpdateEtlLibraries' => true || false,
]);

Parameter Details

Members

AddArguments

Type: Associative array of custom strings keys (GenericString) to strings

The map of arguments to add the map of arguments used to configure the DevEndpoint.

AddPublicKeys

Type: Array of strings

The list of public keys for the DevEndpoint to use.

CustomLibraries

Type: DevEndpointCustomLibraries structure

Custom Python or Java libraries to be loaded in the DevEndpoint.

DeleteArguments

Type: Array of strings

The list of argument keys to be deleted from the map of arguments used to configure the DevEndpoint.

DeletePublicKeys

Type: Array of strings

The list of public keys to be deleted from the DevEndpoint.

EndpointName

Required: Yes
Type: string

The name of the DevEndpoint to be updated.

PublicKey

Type: string

The public key for the DevEndpoint to use.

UpdateEtlLibraries

Type: boolean

True if the list of custom libraries to be loaded in the development endpoint needs to be updated, or False otherwise.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
InvalidInputException:
The input provided was not valid.
ValidationException:
A value could not be validated.

UpdateJob

$result = $client->updateJob([/* ... */]);
$promise = $client->updateJobAsync([/* ... */]);

Updates an existing job definition.

Parameter Syntax

$result = $client->updateJob([
    'JobName' => '<string>', // REQUIRED
    'JobUpdate' => [ // REQUIRED
        'AllocatedCapacity' => <integer>,
        'Command' => [
            'Name' => '<string>',
            'ScriptLocation' => '<string>',
        ],
        'Connections' => [
            'Connections' => ['<string>', ...],
        ],
        'DefaultArguments' => ['<string>', ...],
        'Description' => '<string>',
        'ExecutionProperty' => [
            'MaxConcurrentRuns' => <integer>,
        ],
        'LogUri' => '<string>',
        'MaxCapacity' => <float>,
        'MaxRetries' => <integer>,
        'NotificationProperty' => [
            'NotifyDelayAfter' => <integer>,
        ],
        'NumberOfWorkers' => <integer>,
        'Role' => '<string>',
        'SecurityConfiguration' => '<string>',
        'Timeout' => <integer>,
        'WorkerType' => 'Standard|G.1X|G.2X',
    ],
]);

Parameter Details

Members

JobName

Required: Yes
Type: string

Name of the job definition to update.

JobUpdate

Required: Yes
Type: JobUpdate structure

Specifies the values with which to update the job definition.

Result Syntax

[
    'JobName' => '<string>',
]

Result Details

Members

JobName

Type: string

Returns the name of the updated job definition.

Errors

InvalidInputException:
The input provided was not valid.
EntityNotFoundException:
A specified entity does not exist
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.

UpdatePartition

$result = $client->updatePartition([/* ... */]);
$promise = $client->updatePartitionAsync([/* ... */]);

Updates a partition.

Parameter Syntax

$result = $client->updatePartition([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'PartitionInput' => [ // REQUIRED
        'LastAccessTime' => <integer || string || DateTime>,
        'LastAnalyzedTime' => <integer || string || DateTime>,
        'Parameters' => ['<string>', ...],
        'StorageDescriptor' => [
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>', // REQUIRED
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>', // REQUIRED
                    'SortOrder' => <integer>, // REQUIRED
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'Values' => ['<string>', ...],
    ],
    'PartitionValueList' => ['<string>', ...], // REQUIRED
    'TableName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the partition to be updated resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database in which the table in question resides.

PartitionInput

Required: Yes
Type: PartitionInput structure

The new partition object to which to update the partition.

PartitionValueList

Required: Yes
Type: Array of strings

A list of the values defining the partition.

TableName

Required: Yes
Type: string

The name of the table where the partition to be updated is located.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

UpdateTable

$result = $client->updateTable([/* ... */]);
$promise = $client->updateTableAsync([/* ... */]);

Updates a metadata table in the Data Catalog.

Parameter Syntax

$result = $client->updateTable([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'SkipArchive' => true || false,
    'TableInput' => [ // REQUIRED
        'Description' => '<string>',
        'LastAccessTime' => <integer || string || DateTime>,
        'LastAnalyzedTime' => <integer || string || DateTime>,
        'Name' => '<string>', // REQUIRED
        'Owner' => '<string>',
        'Parameters' => ['<string>', ...],
        'PartitionKeys' => [
            [
                'Comment' => '<string>',
                'Name' => '<string>', // REQUIRED
                'Type' => '<string>',
            ],
            // ...
        ],
        'Retention' => <integer>,
        'StorageDescriptor' => [
            'BucketColumns' => ['<string>', ...],
            'Columns' => [
                [
                    'Comment' => '<string>',
                    'Name' => '<string>', // REQUIRED
                    'Type' => '<string>',
                ],
                // ...
            ],
            'Compressed' => true || false,
            'InputFormat' => '<string>',
            'Location' => '<string>',
            'NumberOfBuckets' => <integer>,
            'OutputFormat' => '<string>',
            'Parameters' => ['<string>', ...],
            'SerdeInfo' => [
                'Name' => '<string>',
                'Parameters' => ['<string>', ...],
                'SerializationLibrary' => '<string>',
            ],
            'SkewedInfo' => [
                'SkewedColumnNames' => ['<string>', ...],
                'SkewedColumnValueLocationMaps' => ['<string>', ...],
                'SkewedColumnValues' => ['<string>', ...],
            ],
            'SortColumns' => [
                [
                    'Column' => '<string>', // REQUIRED
                    'SortOrder' => <integer>, // REQUIRED
                ],
                // ...
            ],
            'StoredAsSubDirectories' => true || false,
        ],
        'TableType' => '<string>',
        'ViewExpandedText' => '<string>',
        'ViewOriginalText' => '<string>',
    ],
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database in which the table resides. For Hive compatibility, this name is entirely lowercase.

SkipArchive

Type: boolean

By default, UpdateTable always creates an archived version of the table before updating it. If skipArchive is set to true, however, UpdateTable does not create the archived version.

TableInput

Required: Yes
Type: TableInput structure

An updated TableInput object to define the metadata table in the catalog.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
GlueEncryptionException:
An encryption operation failed.

UpdateTrigger

$result = $client->updateTrigger([/* ... */]);
$promise = $client->updateTriggerAsync([/* ... */]);

Updates a trigger definition.

Parameter Syntax

$result = $client->updateTrigger([
    'Name' => '<string>', // REQUIRED
    'TriggerUpdate' => [ // REQUIRED
        'Actions' => [
            [
                'Arguments' => ['<string>', ...],
                'JobName' => '<string>',
                'NotificationProperty' => [
                    'NotifyDelayAfter' => <integer>,
                ],
                'SecurityConfiguration' => '<string>',
                'Timeout' => <integer>,
            ],
            // ...
        ],
        'Description' => '<string>',
        'Name' => '<string>',
        'Predicate' => [
            'Conditions' => [
                [
                    'JobName' => '<string>',
                    'LogicalOperator' => 'EQUALS',
                    'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
                ],
                // ...
            ],
            'Logical' => 'AND|ANY',
        ],
        'Schedule' => '<string>',
    ],
]);

Parameter Details

Members

Name

Required: Yes
Type: string

The name of the trigger to update.

TriggerUpdate

Required: Yes
Type: TriggerUpdate structure

The new values with which to update the trigger.

Result Syntax

[
    'Trigger' => [
        'Actions' => [
            [
                'Arguments' => ['<string>', ...],
                'JobName' => '<string>',
                'NotificationProperty' => [
                    'NotifyDelayAfter' => <integer>,
                ],
                'SecurityConfiguration' => '<string>',
                'Timeout' => <integer>,
            ],
            // ...
        ],
        'Description' => '<string>',
        'Id' => '<string>',
        'Name' => '<string>',
        'Predicate' => [
            'Conditions' => [
                [
                    'JobName' => '<string>',
                    'LogicalOperator' => 'EQUALS',
                    'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
                ],
                // ...
            ],
            'Logical' => 'AND|ANY',
        ],
        'Schedule' => '<string>',
        'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
        'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
    ],
]

Result Details

Members

Trigger

Type: Trigger structure

The resulting trigger definition.

Errors

InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
EntityNotFoundException:
A specified entity does not exist
OperationTimeoutException:
The operation timed out.
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.

UpdateUserDefinedFunction

$result = $client->updateUserDefinedFunction([/* ... */]);
$promise = $client->updateUserDefinedFunctionAsync([/* ... */]);

Updates an existing function definition in the Data Catalog.

Parameter Syntax

$result = $client->updateUserDefinedFunction([
    'CatalogId' => '<string>',
    'DatabaseName' => '<string>', // REQUIRED
    'FunctionInput' => [ // REQUIRED
        'ClassName' => '<string>',
        'FunctionName' => '<string>',
        'OwnerName' => '<string>',
        'OwnerType' => 'USER|ROLE|GROUP',
        'ResourceUris' => [
            [
                'ResourceType' => 'JAR|FILE|ARCHIVE',
                'Uri' => '<string>',
            ],
            // ...
        ],
    ],
    'FunctionName' => '<string>', // REQUIRED
]);

Parameter Details

Members

CatalogId

Type: string

The ID of the Data Catalog where the function to be updated is located. If none is supplied, the AWS account ID is used by default.

DatabaseName

Required: Yes
Type: string

The name of the catalog database where the function to be updated is located.

FunctionInput

Required: Yes
Type: UserDefinedFunctionInput structure

A FunctionInput object that re-defines the function in the Data Catalog.

FunctionName

Required: Yes
Type: string

The name of the function.

Result Syntax

[]

Result Details

The results for this operation are always empty.

Errors

EntityNotFoundException:
A specified entity does not exist
InvalidInputException:
The input provided was not valid.
InternalServiceException:
An internal service error occurred.
OperationTimeoutException:
The operation timed out.
GlueEncryptionException:
An encryption operation failed.

Shapes

AccessDeniedException

Description

Access to a resource was denied.

Members

Message

Type: string

A message describing the problem.

Action

Description

Defines an action to be initiated by a trigger.

Members

Arguments

Type: Associative array of custom strings keys (GenericString) to strings

The job arguments used when this trigger fires. For this job run, they replace the default arguments set in the job definition itself.

You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.

For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.

For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

JobName

Type: string

The name of a job to be executed.

NotificationProperty

Type: NotificationProperty structure

Specifies configuration properties of a job run notification.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this action.

Timeout

Type: int

AlreadyExistsException

Description

A resource to be created or added already exists.

Members

Message

Type: string

A message describing the problem.

BatchStopJobRunError

Description

Records an error that occurred when attempting to stop a specified job run.

Members

ErrorDetail

Type: ErrorDetail structure

Specifies details about the error that was encountered.

JobName

Type: string

The name of the job definition used in the job run in question.

JobRunId

Type: string

The JobRunId of the job run in question.

BatchStopJobRunSuccessfulSubmission

Description

Records a successful request to stop a specified JobRun.

Members

JobName

Type: string

The name of the job definition used in the job run that was stopped.

JobRunId

Type: string

The JobRunId of the job run that was stopped.

CatalogEntry

Description

Specifies a table definition in the Data Catalog.

Members

DatabaseName

Required: Yes
Type: string

The database in which the table metadata resides.

TableName

Required: Yes
Type: string

The name of the table in question.

CatalogImportStatus

Description

A structure containing migration status information.

Members

ImportCompleted

Type: boolean

True if the migration has completed, or False otherwise.

ImportTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time that the migration was started.

ImportedBy

Type: string

The name of the person who initiated the migration.

Classifier

Description

Classifiers are triggered during a crawl task. A classifier checks whether a given file is in a format it can handle, and if it is, the classifier creates a schema in the form of a StructType object that matches that data format.

You can use the standard classifiers that AWS Glue supplies, or you can write your own classifiers to best categorize your data sources and specify the appropriate schemas to use for them. A classifier can be a grok classifier, an XML classifier, a JSON classifier, or a custom CSV classifier as specified in one of the fields in the Classifier object.

Members

CsvClassifier

Type: CsvClassifier structure

A CSVClassifier object.

GrokClassifier

Type: GrokClassifier structure

A GrokClassifier object.

JsonClassifier

Type: JsonClassifier structure

A JsonClassifier object.

XMLClassifier

Type: XMLClassifier structure

An XMLClassifier object.

CloudWatchEncryption

Description

Specifies how CloudWatch data should be encrypted.

Members

CloudWatchEncryptionMode

Type: string

The encryption mode to use for CloudWatch data.

KmsKeyArn

Type: string

The AWS ARN of the KMS key to be used to encrypt the data.

CodeGenEdge

Description

Represents a directional edge in a directed acyclic graph (DAG).

Members

Source

Required: Yes
Type: string

The ID of the node at which the edge starts.

Target

Required: Yes
Type: string

The ID of the node at which the edge ends.

TargetParameter

Type: string

The target of the edge.

CodeGenNode

Description

Represents a node in a directed acyclic graph (DAG)

Members

Args

Required: Yes
Type: Array of CodeGenNodeArg structures

Properties of the node, in the form of name-value pairs.

Id

Required: Yes
Type: string

A node identifier that is unique within the node's graph.

LineNumber

Type: int

The line number of the node.

NodeType

Required: Yes
Type: string

The type of node this is.

CodeGenNodeArg

Description

An argument or property of a node.

Members

Name

Required: Yes
Type: string

The name of the argument or property.

Param

Type: boolean

True if the value is used as a parameter.

Value

Required: Yes
Type: string

The value of the argument or property.

Column

Description

A column in a Table.

Members

Comment

Type: string

Free-form text comment.

Name

Required: Yes
Type: string

The name of the Column.

Type

Type: string

The datatype of data in the Column.

ConcurrentModificationException

Description

Two processes are trying to modify a resource simultaneously.

Members

Message

Type: string

A message describing the problem.

ConcurrentRunsExceededException

Description

Too many jobs are being run concurrently.

Members

Message

Type: string

A message describing the problem.

Condition

Description

Defines a condition under which a trigger fires.

Members

JobName

Type: string

The name of the Job to whose JobRuns this condition applies and on which this trigger waits.

LogicalOperator

Type: string

A logical operator.

State

Type: string

The condition state. Currently, the values supported are SUCCEEDED, STOPPED, TIMEOUT and FAILED.

ConditionCheckFailureException

Description

A specified condition was not satisfied.

Members

Message

Type: string

A message describing the problem.

Connection

Description

Defines a connection to a data source.

Members

ConnectionProperties

Type: Associative array of custom strings keys (ConnectionPropertyKey) to strings

These key-value pairs define parameters for the connection:

HOST - The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host.
PORT - The port number, between 1024 and 65535, of the port on which the database host is listening for database connections.
USER_NAME - The name under which to log in to the database. The value string for USER_NAME is "USERNAME".
PASSWORD - A password, if one is used, for the user name.
ENCRYPTED_PASSWORD - When you enable connection password protection by setting ConnectionPasswordEncryption in the Data Catalog encryption settings, this field stores the encrypted password.
JDBC_DRIVER_JAR_URI - The Amazon S3 path of the JAR file that contains the JDBC driver to use.
JDBC_DRIVER_CLASS_NAME - The class name of the JDBC driver to use.
JDBC_ENGINE - The name of the JDBC engine to use.
JDBC_ENGINE_VERSION - The version of the JDBC engine to use.
CONFIG_FILES - (Reserved for future use).
INSTANCE_ID - The instance ID to use.
JDBC_CONNECTION_URL - The URL for the JDBC connection.
JDBC_ENFORCE_SSL - A Boolean string (true, false) specifying whether Secure Sockets Layer (SSL) with hostname matching will be enforced for the JDBC connection on the client. The default is false.

ConnectionType

Type: string

The type of the connection. Currently, only JDBC is supported; SFTP is not supported.

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time that this connection definition was created.

Description

Type: string

The description of the connection.

LastUpdatedBy

Type: string

The user, group, or role that last updated this connection definition.

LastUpdatedTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last time that this connection definition was updated.

MatchCriteria

Type: Array of strings

A list of criteria that can be used in selecting this connection.

Name

Type: string

The name of the connection definition.

PhysicalConnectionRequirements

Type: PhysicalConnectionRequirements structure

A map of physical connection requirements, such as virtual private cloud (VPC) and SecurityGroup, that are needed to make this connection successfully.

ConnectionInput

Description

A structure that is used to specify a connection to create or update.

Members

ConnectionProperties

Required: Yes
Type: Associative array of custom strings keys (ConnectionPropertyKey) to strings

These key-value pairs define parameters for the connection.

ConnectionType

Required: Yes
Type: string

The type of the connection. Currently, only JDBC is supported; SFTP is not supported.

Description

Type: string

The description of the connection.

MatchCriteria

Type: Array of strings

A list of criteria that can be used in selecting this connection.

Name

Required: Yes
Type: string

The name of the connection.

PhysicalConnectionRequirements

Type: PhysicalConnectionRequirements structure

A map of physical connection requirements, such as virtual private cloud (VPC) and SecurityGroup, that are needed to successfully make this connection.

ConnectionPasswordEncryption

Description

The data structure used by the Data Catalog to encrypt the password as part of CreateConnection or UpdateConnection and store it in the ENCRYPTED_PASSWORD field in the connection properties. You can enable catalog encryption or only password encryption.

When a CreationConnection request arrives containing a password, the Data Catalog first encrypts the password using your AWS KMS key. It then encrypts the whole connection object again if catalog encryption is also enabled.

This encryption requires that you set AWS KMS key permissions to enable or restrict access on the password key according to your security requirements. For example, you might want only admin users to have decrypt permission on the password key.

Members

AwsKmsKeyId

Type: string

An AWS KMS key that is used to encrypt the connection password.

If connection password protection is enabled, the caller of CreateConnection and UpdateConnection needs at least kms:Encrypt permission on the specified AWS KMS key, to encrypt passwords before storing them in the Data Catalog.

You can set the decrypt permission to enable or restrict access on the password key according to your security requirements.

ReturnConnectionPasswordEncrypted

Required: Yes
Type: boolean

When the ReturnConnectionPasswordEncrypted flag is set to "true", passwords remain encrypted in the responses of GetConnection and GetConnections. This encryption takes effect independently from catalog encryption.

ConnectionsList

Description

Specifies the connections used by a job.

Members

Connections

Type: Array of strings

A list of connections used by the job.

Crawler

Description

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.

Members

Classifiers

Type: Array of strings

A list of custom classifiers associated with the crawler.

Configuration

Type: string

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

CrawlElapsedTime

Type: long (int|float)

If the crawler is running, contains the total time elapsed since the last crawl began.

CrawlerSecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used by this Crawler.

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time when the crawler was created.

DatabaseName

Type: string

The database where metadata is written by this crawler.

Description

Type: string

A description of the crawler.

LastCrawl

Type: LastCrawlInfo structure

The status of the last crawl, and potentially error information if an error occurred.

LastUpdated

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time the crawler was last updated.

Name

Type: string

The crawler name.

Role

Type: string

The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.

Schedule

Type: Schedule structure

For scheduled crawlers, the schedule when the crawler runs.

SchemaChangePolicy

Type: SchemaChangePolicy structure

Sets the behavior when the crawler finds a changed or deleted object.

State

Type: string

Indicates whether the crawler is running, or whether a run is pending.

TablePrefix

Type: string

The prefix added to the names of tables that are created.

Targets

Type: CrawlerTargets structure

A collection of targets to crawl.

Version

Type: long (int|float)

The version of the crawler.

CrawlerMetrics

Description

Metrics for a specified crawler.

Members

CrawlerName

Type: string

The name of the crawler.

LastRuntimeSeconds

Type: double

The duration of the crawler's most recent run, in seconds.

MedianRuntimeSeconds

Type: double

The median duration of this crawler's runs, in seconds.

StillEstimating

Type: boolean

True if the crawler is still estimating how long it will take to complete this run.

TablesCreated

Type: int

The number of tables created by this crawler.

TablesDeleted

Type: int

The number of tables deleted by this crawler.

TablesUpdated

Type: int

The number of tables updated by this crawler.

TimeLeftSeconds

Type: double

The estimated time left to complete a running crawl.

CrawlerNotRunningException

Description

The specified crawler is not running.

Members

Message

Type: string

A message describing the problem.

CrawlerRunningException

Description

The operation cannot be performed because the crawler is already running.

Members

Message

Type: string

A message describing the problem.

CrawlerStoppingException

Description

The specified crawler is stopping.

Members

Message

Type: string

A message describing the problem.

CrawlerTargets

Description

Specifies data stores to crawl.

Members

DynamoDBTargets

Type: Array of DynamoDBTarget structures

Specifies DynamoDB targets.

JdbcTargets

Type: Array of JdbcTarget structures

Specifies JDBC targets.

S3Targets

Type: Array of S3Target structures

Specifies Amazon S3 targets.

CreateCsvClassifierRequest

Description

Specifies a custom CSV classifier for CreateClassifier to create.

Members

AllowSingleColumn

Type: boolean

Enables the processing of files that contain only one column.

ContainsHeader

Type: string

Indicates whether the CSV file contains a header.

Delimiter

Type: string

A custom symbol to denote what separates each column entry in the row.

DisableValueTrimming

Type: boolean

Specifies not to trim values before identifying the type of column values. The default value is true.

Header

Type: Array of strings

A list of strings representing column names.

Name

Required: Yes
Type: string

The name of the classifier.

QuoteSymbol

Type: string

A custom symbol to denote what combines content into a single column value. Must be different from the column delimiter.

CreateGrokClassifierRequest

Description

Specifies a grok classifier for CreateClassifier to create.

Members

Classification

Required: Yes
Type: string

An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.

CustomPatterns

Type: string

Optional custom grok patterns used by this classifier.

GrokPattern

Required: Yes
Type: string

The grok pattern used by this classifier.

Name

Required: Yes
Type: string

The name of the new classifier.

CreateJsonClassifierRequest

Description

Specifies a JSON classifier for CreateClassifier to create.

Members

JsonPath

Required: Yes
Type: string

A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.

Name

Required: Yes
Type: string

The name of the classifier.

CreateXMLClassifierRequest

Description

Specifies an XML classifier for CreateClassifier to create.

Members

Classification

Required: Yes
Type: string

An identifier of the data format that the classifier matches.

Name

Required: Yes
Type: string

The name of the classifier.

RowTag

Type: string

The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by />). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).

CsvClassifier

Description

A classifier for custom CSV content.

Members

AllowSingleColumn

Type: boolean

Enables the processing of files that contain only one column.

ContainsHeader

Type: string

Indicates whether the CSV file contains a header.

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was registered.

Delimiter

Type: string

A custom symbol to denote what separates each column entry in the row.

DisableValueTrimming

Type: boolean

Specifies not to trim values before identifying the type of column values. The default value is true.

Header

Type: Array of strings

A list of strings representing column names.

LastUpdated

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was last updated.

Name

Required: Yes
Type: string

The name of the classifier.

QuoteSymbol

Type: string

A custom symbol to denote what combines content into a single column value. Must be different from the column delimiter.

Version

Type: long (int|float)

The version of this classifier.

DataCatalogEncryptionSettings

Description

Contains configuration information for maintaining Data Catalog security.

Members

ConnectionPasswordEncryption

Type: ConnectionPasswordEncryption structure

When connection password protection is enabled, the Data Catalog uses a customer-provided key to encrypt the password as part of CreateConnection or UpdateConnection and store it in the ENCRYPTED_PASSWORD field in the connection properties. You can enable catalog encryption or only password encryption.

EncryptionAtRest

Type: EncryptionAtRest structure

Specifies the encryption-at-rest configuration for the Data Catalog.

Database

Description

The Database object represents a logical grouping of tables that may reside in a Hive metastore or an RDBMS.

Members

CreateTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time at which the metadata database was created in the catalog.

Description

Type: string

Description of the database.

LocationUri

Type: string

The location of the database (for example, an HDFS path).

Name

Required: Yes
Type: string

Name of the database. For Hive compatibility, this is folded to lowercase when it is stored.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

These key-value pairs define parameters and properties of the database.

DatabaseInput

Description

The structure used to create or update a database.

Members

Description

Type: string

Description of the database

LocationUri

Type: string

The location of the database (for example, an HDFS path).

Name

Required: Yes
Type: string

Name of the database. For Hive compatibility, this is folded to lowercase when it is stored.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

Thes key-value pairs define parameters and properties of the database.

DevEndpoint

Description

A development endpoint where a developer can remotely debug ETL scripts.

Members

Arguments

Type: Associative array of custom strings keys (GenericString) to strings

A map of arguments used to configure the DevEndpoint.

Note that currently, we only support "--enable-glue-datacatalog": "" as a valid argument.

AvailabilityZone

Type: string

The AWS availability zone where this DevEndpoint is located.

CreatedTimestamp

Type: timestamp (string|DateTime or anything parsable by strtotime)

The point in time at which this DevEndpoint was created.

EndpointName

Type: string

The name of the DevEndpoint.

ExtraJarsS3Path

Type: string

Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.

Please note that only pure Java/Scala libraries can currently be used on a DevEndpoint.

ExtraPythonLibsS3Path

Type: string

Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

FailureReason

Type: string

The reason for a current failure in this DevEndpoint.

LastModifiedTimestamp

Type: timestamp (string|DateTime or anything parsable by strtotime)

The point in time at which this DevEndpoint was last modified.

LastUpdateStatus

Type: string

The status of the last update.

NumberOfNodes

Type: int

The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

PrivateAddress

Type: string

A private IP address to access the DevEndpoint within a VPC, if the DevEndpoint is created within one. The PrivateAddress field is present only when you create the DevEndpoint within your virtual private cloud (VPC).

PublicAddress

Type: string

The public IP address used by this DevEndpoint. The PublicAddress field is present only when you create a non-VPC (virtual private cloud) DevEndpoint.

PublicKey

Type: string

The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility, as the recommended attribute to use is public keys.

PublicKeys

Type: Array of strings

RoleArn

Type: string

The AWS ARN of the IAM role used in this DevEndpoint.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this DevEndpoint.

SecurityGroupIds

Type: Array of strings

A list of security group identifiers used in this DevEndpoint.

Status

Type: string

The current status of this DevEndpoint.

SubnetId

Type: string

The subnet ID for this DevEndpoint.

VpcId

Type: string

The ID of the virtual private cloud (VPC) used by this DevEndpoint.

YarnEndpointAddress

Type: string

The YARN endpoint address used by this DevEndpoint.

ZeppelinRemoteSparkInterpreterPort

Type: int

The Apache Zeppelin port for the remote Apache Spark interpreter.

DevEndpointCustomLibraries

Description

Custom libraries to be loaded into a DevEndpoint.

Members

ExtraJarsS3Path

Type: string

Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.

Please note that only pure Java/Scala libraries can currently be used on a DevEndpoint.

ExtraPythonLibsS3Path

Type: string

Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

DynamoDBTarget

Description

Specifies a DynamoDB table to crawl.

Members

Path

Type: string

The name of the DynamoDB table to crawl.

EncryptionAtRest

Description

Specifies the encryption-at-rest configuration for the Data Catalog.

Members

CatalogEncryptionMode

Required: Yes
Type: string

The encryption-at-rest mode for encrypting Data Catalog data.

SseAwsKmsKeyId

Type: string

The ID of the AWS KMS key to use for encryption at rest.

EncryptionConfiguration

Description

Specifies an encryption configuration.

Members

CloudWatchEncryption

Type: CloudWatchEncryption structure

The encryption configuration for CloudWatch.

JobBookmarksEncryption

Type: JobBookmarksEncryption structure

The encryption configuration for Job Bookmarks.

S3Encryption

Type: Array of S3Encryption structures

The encryption configuration for S3 data.

EntityNotFoundException

Description

A specified entity does not exist

Members

Message

Type: string

A message describing the problem.

ErrorDetail

Description

Contains details about an error.

Members

ErrorCode

Type: string

The code associated with this error.

ErrorMessage

Type: string

A message describing the error.

ExecutionProperty

Description

An execution property of a job.

Members

MaxConcurrentRuns

Type: int

The maximum number of concurrent runs allowed for the job. The default is 1. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.

GetConnectionsFilter

Description

Filters the connection definitions that are returned by the GetConnections API operation.

Members

ConnectionType

Type: string

The type of connections to return. Currently, only JDBC is supported; SFTP is not supported.

MatchCriteria

Type: Array of strings

A criteria string that must match the criteria recorded in the connection definition for that connection definition to be returned.

GlueEncryptionException

Description

An encryption operation failed.

Members

Message

Type: string

A message describing the problem.

GrokClassifier

Description

A classifier that uses grok patterns.

Members

Classification

Required: Yes
Type: string

An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was registered.

CustomPatterns

Type: string

Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifers.

GrokPattern

Required: Yes
Type: string

The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifers.

LastUpdated

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was last updated.

Name

Required: Yes
Type: string

The name of the classifier.

Version

Type: long (int|float)

The version of this classifier.

IdempotentParameterMismatchException

Description

The same unique identifier was associated with two different records.

Members

Message

Type: string

A message describing the problem.

InternalServiceException

Description

An internal service error occurred.

Members

Message

Type: string

A message describing the problem.

InvalidInputException

Description

The input provided was not valid.

Members

Message

Type: string

A message describing the problem.

JdbcTarget

Description

Specifies a JDBC data store to crawl.

Members

ConnectionName

Type: string

The name of the connection to use to connect to the JDBC target.

Exclusions

Type: Array of strings

A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler.

Path

Type: string

The path of the JDBC target.

Job

Description

Specifies a job definition.

Members

AllocatedCapacity

Type: int

This field is deprecated, use MaxCapacity instead.

The number of AWS Glue data processing units (DPUs) allocated to runs of this job. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.

Command

Type: JobCommand structure

The JobCommand that executes this job.

Connections

Type: ConnectionsList structure

The connections used for this job.

CreatedOn

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time and date that this job definition was created.

DefaultArguments

Type: Associative array of custom strings keys (GenericString) to strings

The default arguments for this job, specified as name-value pairs.

You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.

For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.

For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

Description

Type: string

Description of the job being defined.

ExecutionProperty

Type: ExecutionProperty structure

An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job.

LastModifiedOn

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last point in time when this job definition was modified.

LogUri

Type: string

This field is reserved for future use.

MaxCapacity

Type: double

Do not set Max Capacity if using WorkerType and NumberOfWorkers.

The value that can be allocated for MaxCapacity depends on whether you are running a python shell job, or an Apache Spark ETL job:

When you specify a python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
When you specify an Apache Spark ETL job (JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

MaxRetries

Type: int

The maximum number of times to retry this job after a JobRun fails.

Name

Type: string

The name you assign to this job definition.

NotificationProperty

Type: NotificationProperty structure

Specifies configuration properties of a job notification.

NumberOfWorkers

Type: int

The number of workers of a defined workerType that are allocated when a job runs.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

Role

Type: string

The name or ARN of the IAM role associated with this job.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this job.

Timeout

Type: int

The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

WorkerType

Type: string

The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

JobBookmarkEntry

Description

Defines a point which a job can resume processing.

Members

Attempt

Type: int

The attempt ID number.

JobBookmark

Type: string

The bookmark itself.

JobName

Type: string

Name of the job in question.

Run

Type: int

The run ID number.

Version

Type: int

Version of the job.

JobBookmarksEncryption

Description

Specifies how Job bookmark data should be encrypted.

Members

JobBookmarksEncryptionMode

Type: string

The encryption mode to use for Job bookmarks data.

KmsKeyArn

Type: string

The AWS ARN of the KMS key to be used to encrypt the data.

JobCommand

Description

Specifies code executed when a job is run.

Members

Name

Type: string

The name of the job command: this must be glueetl, for an Apache Spark ETL job, or pythonshell, for a Python shell job.

ScriptLocation

Type: string

Specifies the S3 path to a script that executes a job (required).

JobRun

Description

Contains information about a job run.

Members

AllocatedCapacity

Type: int

This field is deprecated, use MaxCapacity instead.

The number of AWS Glue data processing units (DPUs) allocated to this JobRun. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.

Arguments

Type: Associative array of custom strings keys (GenericString) to strings

The job arguments associated with this run. For this job run, they replace the default arguments set in the job definition itself.

You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.

For information about how to specify and consume your own job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.

For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

Attempt

Type: int

The number of the attempt to run this job.

CompletedOn

Type: timestamp (string|DateTime or anything parsable by strtotime)

The date and time this job run completed.

ErrorMessage

Type: string

An error message associated with this job run.

ExecutionTime

Type: int

The amount of time (in seconds) that the job run consumed resources.

Id

Type: string

The ID of this job run.

JobName

Type: string

The name of the job definition being used in this run.

JobRunState

Type: string

The current state of the job run.

LastModifiedOn

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last time this job run was modified.

LogGroupName

Type: string

The name of the log group for secure logging, that can be server-side encrypted in CloudWatch using KMS. This name can be /aws-glue/jobs/, in which case the default encryption is NONE. If you add a role name and SecurityConfiguration name (in other words, /aws-glue/jobs-yourRoleName-yourSecurityConfigurationName/), then that security configuration will be used to encrypt the log group.

MaxCapacity

Type: double

Do not set Max Capacity if using WorkerType and NumberOfWorkers.

The value that can be allocated for MaxCapacity depends on whether you are running a python shell job, or an Apache Spark ETL job:

When you specify a python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
When you specify an Apache Spark ETL job (JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

NotificationProperty

Type: NotificationProperty structure

Specifies configuration properties of a job run notification.

NumberOfWorkers

Type: int

The number of workers of a defined workerType that are allocated when a job runs.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

PredecessorRuns

Type: Array of Predecessor structures

A list of predecessors to this job run.

PreviousRunId

Type: string

The ID of the previous run of this job. For example, the JobRunId specified in the StartJobRun action.

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this job run.

StartedOn

Type: timestamp (string|DateTime or anything parsable by strtotime)

The date and time at which this job run was started.

Timeout

Type: int

TriggerName

Type: string

The name of the trigger that started this job run.

WorkerType

Type: string

The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

JobUpdate

Description

Specifies information used to update an existing job definition. Note that the previous job definition will be completely overwritten by this information.

Members

AllocatedCapacity

Type: int

This field is deprecated. Use MaxCapacity instead.

Command

Type: JobCommand structure

The JobCommand that executes this job (required).

Connections

Type: ConnectionsList structure

The connections used for this job.

DefaultArguments

Type: Associative array of custom strings keys (GenericString) to strings

The default arguments for this job.

You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.

For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.

For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.

Description

Type: string

Description of the job being defined.

ExecutionProperty

Type: ExecutionProperty structure

An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job.

LogUri

Type: string

This field is reserved for future use.

MaxCapacity

Type: double

Do not set Max Capacity if using WorkerType and NumberOfWorkers.

The value that can be allocated for MaxCapacity depends on whether you are running a python shell job, or an Apache Spark ETL job:

When you specify a python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.
When you specify an Apache Spark ETL job (JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

MaxRetries

Type: int

The maximum number of times to retry this job if it fails.

NotificationProperty

Type: NotificationProperty structure

Specifies configuration properties of a job notification.

NumberOfWorkers

Type: int

The number of workers of a defined workerType that are allocated when a job runs.

The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

Role

Type: string

The name or ARN of the IAM role associated with this job (required).

SecurityConfiguration

Type: string

The name of the SecurityConfiguration structure to be used with this job.

Timeout

Type: int

The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).

WorkerType

Type: string

The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.

For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.
For the G.1X worker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker.
For the G.2X worker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.

JsonClassifier

Description

A classifier for JSON content.

Members

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was registered.

JsonPath

Required: Yes
Type: string

A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.

LastUpdated

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was last updated.

Name

Required: Yes
Type: string

The name of the classifier.

Version

Type: long (int|float)

The version of this classifier.

LastCrawlInfo

Description

Status and error information about the most recent crawl.

Members

ErrorMessage

Type: string

If an error occurred, the error information about the last crawl.

LogGroup

Type: string

The log group for the last crawl.

LogStream

Type: string

The log stream for the last crawl.

MessagePrefix

Type: string

The prefix for a message about this crawl.

StartTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time at which the crawl started.

Status

Type: string

Status of the last crawl.

Location

Description

The location of resources.

Members

DynamoDB

Type: Array of CodeGenNodeArg structures

A DynamoDB Table location.

Jdbc

Type: Array of CodeGenNodeArg structures

A JDBC location.

S3

Type: Array of CodeGenNodeArg structures

An Amazon S3 location.

MappingEntry

Description

Defines a mapping.

Members

SourcePath

Type: string

The source path.

SourceTable

Type: string

The name of the source table.

SourceType

Type: string

The source type.

TargetPath

Type: string

The target path.

TargetTable

Type: string

The target table.

TargetType

Type: string

The target type.

NoScheduleException

Description

There is no applicable schedule.

Members

Message

Type: string

A message describing the problem.

NotificationProperty

Description

Specifies configuration properties of a notification.

Members

NotifyDelayAfter

Type: int

After a job run starts, the number of minutes to wait before sending a job run delay notification.

OperationTimeoutException

Description

The operation timed out.

Members

Message

Type: string

A message describing the problem.

Order

Description

Specifies the sort order of a sorted column.

Members

Column

Required: Yes
Type: string

The name of the column.

SortOrder

Required: Yes
Type: int

Indicates that the column is sorted in ascending order (== 1), or in descending order (==0).

Partition

Description

Represents a slice of table data.

Members

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time at which the partition was created.

DatabaseName

Type: string

The name of the catalog database where the table in question is located.

LastAccessTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last time at which the partition was accessed.

LastAnalyzedTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last time at which column statistics were computed for this partition.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

These key-value pairs define partition parameters.

StorageDescriptor

Type: StorageDescriptor structure

Provides information about the physical location where the partition is stored.

TableName

Type: string

The name of the table in question.

Values

Type: Array of strings

The values of the partition.

PartitionError

Description

Contains information about a partition error.

Members

ErrorDetail

Type: ErrorDetail structure

Details about the partition error.

PartitionValues

Type: Array of strings

The values that define the partition.

PartitionInput

Description

The structure used to create and update a partion.

Members

LastAccessTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last time at which the partition was accessed.

LastAnalyzedTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The last time at which column statistics were computed for this partition.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

These key-value pairs define partition parameters.

StorageDescriptor

Type: StorageDescriptor structure

Provides information about the physical location where the partition is stored.

Values

Type: Array of strings

The values of the partition. Although this parameter is not required by the SDK, you must specify this parameter for a valid input.

PartitionValueList

Description

Contains a list of values defining partitions.

Members

Values

Required: Yes
Type: Array of strings

The list of values.

PhysicalConnectionRequirements

Description

Specifies the physical requirements for a connection.

Members

AvailabilityZone

Type: string

The connection's Availability Zone. This field is redundant because the specified subnet implies the Availability Zone to be used. Currently the field must be populated, but it will be deprecated in the future.

SecurityGroupIdList

Type: Array of strings

The security group ID list used by the connection.

SubnetId

Type: string

The subnet ID used by the connection.

Predecessor

Description

A job run that was used in the predicate of a conditional trigger that triggered this job run.

Members

JobName

Type: string

The name of the job definition used by the predecessor job run.

RunId

Type: string

The job-run ID of the predecessor job run.

Predicate

Description

Defines the predicate of the trigger, which determines when it fires.

Members

Conditions

Type: Array of Condition structures

A list of the conditions that determine when the trigger will fire.

Logical

Type: string

Optional field if only one condition is listed. If multiple conditions are listed, then this field is required.

ResourceNumberLimitExceededException

Description

A resource numerical limit was exceeded.

Members

Message

Type: string

A message describing the problem.

ResourceUri

Description

URIs for function resources.

Members

ResourceType

Type: string

The type of the resource.

Uri

Type: string

The URI for accessing the resource.

S3Encryption

Description

Specifies how S3 data should be encrypted.

Members

KmsKeyArn

Type: string

The AWS ARN of the KMS key to be used to encrypt the data.

S3EncryptionMode

Type: string

The encryption mode to use for S3 data.

S3Target

Description

Specifies a data store in Amazon S3.

Members

Exclusions

Type: Array of strings

A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler.

Path

Type: string

The path to the Amazon S3 target.

Schedule

Description

A scheduling object using a cron statement to schedule an event.

Members

ScheduleExpression

Type: string

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

State

Type: string

The state of the schedule.

SchedulerNotRunningException

Description

The specified scheduler is not running.

Members

Message

Type: string

A message describing the problem.

SchedulerRunningException

Description

The specified scheduler is already running.

Members

Message

Type: string

A message describing the problem.

SchedulerTransitioningException

Description

The specified scheduler is transitioning.

Members

Message

Type: string

A message describing the problem.

SchemaChangePolicy

Description

Crawler policy for update and deletion behavior.

Members

DeleteBehavior

Type: string

The deletion behavior when the crawler finds a deleted object.

UpdateBehavior

Type: string

The update behavior when the crawler finds a changed schema.

SecurityConfiguration

Description

Specifies a security configuration.

Members

CreatedTimeStamp

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time at which this security configuration was created.

EncryptionConfiguration

Type: EncryptionConfiguration structure

The encryption configuration associated with this security configuration.

Name

Type: string

The name of the security configuration.

Segment

Description

Defines a non-overlapping region of a table's partitions, allowing multiple requests to be executed in parallel.

Members

SegmentNumber

Required: Yes
Type: int

The zero-based index number of the this segment. For example, if the total number of segments is 4, SegmentNumber values will range from zero through three.

TotalSegments

Required: Yes
Type: int

The total numer of segments.

SerDeInfo

Description

Information about a serialization/deserialization program (SerDe) which serves as an extractor and loader.

Members

Name

Type: string

Name of the SerDe.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

These key-value pairs define initialization parameters for the SerDe.

SerializationLibrary

Type: string

Usually the class that implements the SerDe. An example is: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe.

SkewedInfo

Description

Specifies skewed values in a table. Skewed are ones that occur with very high frequency.

Members

SkewedColumnNames

Type: Array of strings

A list of names of columns that contain skewed values.

SkewedColumnValueLocationMaps

Type: Associative array of custom strings keys (ColumnValuesString) to strings

A mapping of skewed values to the columns that contain them.

SkewedColumnValues

Type: Array of strings

A list of values that appear so frequently as to be considered skewed.

StorageDescriptor

Description

Describes the physical storage of table data.

Members

BucketColumns

Type: Array of strings

A list of reducer grouping columns, clustering columns, and bucketing columns in the table.

Columns

Type: Array of Column structures

A list of the Columns in the table.

Compressed

Type: boolean

True if the data in the table is compressed, or False if not.

InputFormat

Type: string

The input format: SequenceFileInputFormat (binary), or TextInputFormat, or a custom format.

Location

Type: string

The physical location of the table. By default this takes the form of the warehouse location, followed by the database location in the warehouse, followed by the table name.

NumberOfBuckets

Type: int

Must be specified if the table contains any dimension columns.

OutputFormat

Type: string

The output format: SequenceFileOutputFormat (binary), or IgnoreKeyTextOutputFormat, or a custom format.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

User-supplied properties in key-value form.

SerdeInfo

Type: SerDeInfo structure

Serialization/deserialization (SerDe) information.

SkewedInfo

Type: SkewedInfo structure

Information about values that appear very frequently in a column (skewed values).

SortColumns

Type: Array of Order structures

A list specifying the sort order of each bucket in the table.

StoredAsSubDirectories

Type: boolean

True if the table data is stored in subdirectories, or False if not.

Table

Description

Represents a collection of related data organized in columns and rows.

Members

CreateTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

Time when the table definition was created in the Data Catalog.

CreatedBy

Type: string

Person or entity who created the table.

DatabaseName

Type: string

Name of the metadata database where the table metadata resides. For Hive compatibility, this must be all lowercase.

Description

Type: string

Description of the table.

LastAccessTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

Last time the table was accessed. This is usually taken from HDFS, and may not be reliable.

LastAnalyzedTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

Last time column statistics were computed for this table.

Name

Required: Yes
Type: string

Name of the table. For Hive compatibility, this must be entirely lowercase.

Owner

Type: string

Owner of the table.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

These key-value pairs define properties associated with the table.

PartitionKeys

Type: Array of Column structures

A list of columns by which the table is partitioned. Only primitive types are supported as partition keys.

When creating a table used by Athena, and you do not specify any partitionKeys, you must at least set the value of partitionKeys to an empty list. For example:

"PartitionKeys": []

Retention

Type: int

Retention time for this table.

StorageDescriptor

Type: StorageDescriptor structure

A storage descriptor containing information about the physical storage of this table.

TableType

Type: string

The type of this table (EXTERNAL_TABLE, VIRTUAL_VIEW, etc.).

UpdateTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

Last time the table was updated.

ViewExpandedText

Type: string

If the table is a view, the expanded text of the view; otherwise null.

ViewOriginalText

Type: string

If the table is a view, the original text of the view; otherwise null.

TableError

Description

An error record for table operations.

Members

ErrorDetail

Type: ErrorDetail structure

Detail about the error.

TableName

Type: string

Name of the table. For Hive compatibility, this must be entirely lowercase.

TableInput

Description

Structure used to create or update the table.

Members

Description

Type: string

Description of the table.

LastAccessTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

Last time the table was accessed.

LastAnalyzedTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

Last time column statistics were computed for this table.

Name

Required: Yes
Type: string

Name of the table. For Hive compatibility, this is folded to lowercase when it is stored.

Owner

Type: string

Owner of the table.

Parameters

Type: Associative array of custom strings keys (KeyString) to strings

These key-value pairs define properties associated with the table.

PartitionKeys

Type: Array of Column structures

A list of columns by which the table is partitioned. Only primitive types are supported as partition keys.

When creating a table used by Athena, and you do not specify any partitionKeys, you must at least set the value of partitionKeys to an empty list. For example:

"PartitionKeys": []

Retention

Type: int

Retention time for this table.

StorageDescriptor

Type: StorageDescriptor structure

A storage descriptor containing information about the physical storage of this table.

TableType

Type: string

The type of this table (EXTERNAL_TABLE, VIRTUAL_VIEW, etc.).

ViewExpandedText

Type: string

If the table is a view, the expanded text of the view; otherwise null.

ViewOriginalText

Type: string

If the table is a view, the original text of the view; otherwise null.

TableVersion

Description

Specifies a version of a table.

Members

Table

Type: Table structure

The table in question

VersionId

Type: string

The ID value that identifies this table version. A VersionId is a string representation of an integer. Each version is incremented by 1.

TableVersionError

Description

An error record for table-version operations.

Members

ErrorDetail

Type: ErrorDetail structure

Detail about the error.

TableName

Type: string

The name of the table in question.

VersionId

Type: string

The ID value of the version in question. A VersionID is a string representation of an integer. Each version is incremented by 1.

Trigger

Description

Information about a specific trigger.

Members

Actions

Type: Array of Action structures

The actions initiated by this trigger.

Description

Type: string

A description of this trigger.

Id

Type: string

Reserved for future use.

Name

Type: string

Name of the trigger.

Predicate

Type: Predicate structure

The predicate of this trigger, which defines when it will fire.

Schedule

Type: string

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

State

Type: string

The current state of the trigger.

Type

Type: string

The type of trigger that this is.

TriggerUpdate

Description

A structure used to provide information used to update a trigger. This object will update the the previous trigger definition by overwriting it completely.

Members

Actions

Type: Array of Action structures

The actions initiated by this trigger.

Description

Type: string

A description of this trigger.

Name

Type: string

Reserved for future use.

Predicate

Type: Predicate structure

The predicate of this trigger, which defines when it will fire.

Schedule

Type: string

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

UpdateCsvClassifierRequest

Description

Specifies a custom CSV classifier to be updated.

Members

AllowSingleColumn

Type: boolean

Enables the processing of files that contain only one column.

ContainsHeader

Type: string

Indicates whether the CSV file contains a header.

Delimiter

Type: string

A custom symbol to denote what separates each column entry in the row.

DisableValueTrimming

Type: boolean

Specifies not to trim values before identifying the type of column values. The default value is true.

Header

Type: Array of strings

A list of strings representing column names.

Name

Required: Yes
Type: string

The name of the classifier.

QuoteSymbol

Type: string

A custom symbol to denote what combines content into a single column value. Must be different from the column delimiter.

UpdateGrokClassifierRequest

Description

Specifies a grok classifier to update when passed to UpdateClassifier.

Members

Classification

Type: string

An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.

CustomPatterns

Type: string

Optional custom grok patterns used by this classifier.

GrokPattern

Type: string

The grok pattern used by this classifier.

Name

Required: Yes
Type: string

The name of the GrokClassifier.

UpdateJsonClassifierRequest

Description

Specifies a JSON classifier to be updated.

Members

JsonPath

Type: string

A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.

Name

Required: Yes
Type: string

The name of the classifier.

UpdateXMLClassifierRequest

Description

Specifies an XML classifier to be updated.

Members

Classification

Type: string

An identifier of the data format that the classifier matches.

Name

Required: Yes
Type: string

The name of the classifier.

RowTag

Type: string

UserDefinedFunction

Description

Represents the equivalent of a Hive user-defined function (UDF) definition.

Members

ClassName

Type: string

The Java class that contains the function code.

CreateTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time at which the function was created.

FunctionName

Type: string

The name of the function.

OwnerName

Type: string

The owner of the function.

OwnerType

Type: string

The owner type.

ResourceUris

Type: Array of ResourceUri structures

The resource URIs for the function.

UserDefinedFunctionInput

Description

A structure used to create or updata a user-defined function.

Members

ClassName

Type: string

The Java class that contains the function code.

FunctionName

Type: string

The name of the function.

OwnerName

Type: string

The owner of the function.

OwnerType

Type: string

The owner type.

ResourceUris

Type: Array of ResourceUri structures

The resource URIs for the function.

ValidationException

Description

A value could not be validated.

Members

Message

Type: string

A message describing the problem.

VersionMismatchException

Description

There was a version conflict.

Members

Message

Type: string

A message describing the problem.

XMLClassifier

Description

A classifier for XML content.

Members

Classification

Required: Yes
Type: string

An identifier of the data format that the classifier matches.

CreationTime

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was registered.

LastUpdated

Type: timestamp (string|DateTime or anything parsable by strtotime)

The time this classifier was last updated.

Name

Required: Yes
Type: string

The name of the classifier.

RowTag

Type: string

Version

Type: long (int|float)

The version of this classifier.

Namespaces

Classes

Interfaces

Traits

Exceptions

Functions

AWS Glue 2017-03-31

Operation Summary

Paginators

Operations

BatchCreatePartition

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchDeleteConnection

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchDeletePartition

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchDeleteTable

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchDeleteTableVersion

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchGetCrawlers

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchGetDevEndpoints

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchGetJobs

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details

Members

Errors

BatchGetPartition

Parameter Syntax

Parameter Details

Members

Result Syntax

Result Details