AWS Glue 2017-03-31
- Client: Aws\Glue\GlueClient
- Service ID: glue
- Version: 2017-03-31
This page describes the parameters and results for the operations of the AWS Glue (2017-03-31), and shows how to use the Aws\Glue\GlueClient object to call the described operations. This documentation is specific to the 2017-03-31 API version of the service.
Operation Summary
Each of the following operations can be created from a client using
$client->getCommand('CommandName'), where "CommandName" is the
name of one of the following operations. Note: a command is a value that
encapsulates an operation and the parameters used to create an HTTP request.
You can also create and send a command immediately using the magic methods
available on a client object: $client->commandName(/* parameters */).
You can send the command asynchronously (returning a promise) by appending the
word "Async" to the operation name: $client->commandNameAsync(/* parameters */).
- BatchCreatePartition ( array $params = [] )
Creates one or more partitions in a batch operation.
- BatchDeleteConnection ( array $params = [] )
Deletes a list of connection definitions from the Data Catalog.
- BatchDeletePartition ( array $params = [] )
Deletes one or more partitions in a batch operation.
- BatchDeleteTable ( array $params = [] )
Deletes multiple tables at once.
- BatchDeleteTableVersion ( array $params = [] )
Deletes a specified batch of versions of a table.
- BatchGetCrawlers ( array $params = [] )
Returns a list of resource metadata for a given list of crawler names.
- BatchGetDevEndpoints ( array $params = [] )
Returns a list of resource metadata for a given list of DevEndpoint names.
- BatchGetJobs ( array $params = [] )
Returns a list of resource metadata for a given list of job names.
- BatchGetPartition ( array $params = [] )
Retrieves partitions in a batch request.
- BatchGetTriggers ( array $params = [] )
Returns a list of resource metadata for a given list of trigger names.
- BatchGetWorkflows ( array $params = [] )
Returns a list of resource metadata for a given list of workflow names.
- BatchStopJobRun ( array $params = [] )
Stops one or more job runs for a specified job definition.
- CreateClassifier ( array $params = [] )
Creates a classifier in the user's account.
- CreateConnection ( array $params = [] )
Creates a connection definition in the Data Catalog.
- CreateCrawler ( array $params = [] )
Creates a new crawler with specified targets, role, configuration, and optional schedule.
- CreateDatabase ( array $params = [] )
Creates a new database in a Data Catalog.
- CreateDevEndpoint ( array $params = [] )
Creates a new DevEndpoint.
- CreateJob ( array $params = [] )
Creates a new job definition.
- CreatePartition ( array $params = [] )
Creates a new partition.
- CreateScript ( array $params = [] )
Transforms a directed acyclic graph (DAG) into code.
- CreateSecurityConfiguration ( array $params = [] )
Creates a new security configuration.
- CreateTable ( array $params = [] )
Creates a new table definition in the Data Catalog.
- CreateTrigger ( array $params = [] )
Creates a new trigger.
- CreateUserDefinedFunction ( array $params = [] )
Creates a new function definition in the Data Catalog.
- CreateWorkflow ( array $params = [] )
Creates a new workflow.
- DeleteClassifier ( array $params = [] )
Removes a classifier from the Data Catalog.
- DeleteConnection ( array $params = [] )
Deletes a connection from the Data Catalog.
- DeleteCrawler ( array $params = [] )
Removes a specified crawler from the AWS Glue Data Catalog, unless the crawler state is RUNNING.
- DeleteDatabase ( array $params = [] )
Removes a specified Database from a Data Catalog.
- DeleteDevEndpoint ( array $params = [] )
Deletes a specified DevEndpoint.
- DeleteJob ( array $params = [] )
Deletes a specified job definition.
- DeletePartition ( array $params = [] )
Deletes a specified partition.
- DeleteResourcePolicy ( array $params = [] )
Deletes a specified policy.
- DeleteSecurityConfiguration ( array $params = [] )
Deletes a specified security configuration.
- DeleteTable ( array $params = [] )
Removes a table definition from the Data Catalog.
- DeleteTableVersion ( array $params = [] )
Deletes a specified version of a table.
- DeleteTrigger ( array $params = [] )
Deletes a specified trigger.
- DeleteUserDefinedFunction ( array $params = [] )
Deletes an existing function definition from the Data Catalog.
- DeleteWorkflow ( array $params = [] )
Deletes a workflow.
- GetCatalogImportStatus ( array $params = [] )
Retrieves the status of a migration operation.
- GetClassifier ( array $params = [] )
Retrieve a classifier by name.
- GetClassifiers ( array $params = [] )
Lists all classifier objects in the Data Catalog.
- GetConnection ( array $params = [] )
Retrieves a connection definition from the Data Catalog.
- GetConnections ( array $params = [] )
Retrieves a list of connection definitions from the Data Catalog.
- GetCrawler ( array $params = [] )
Retrieves metadata for a specified crawler.
- GetCrawlerMetrics ( array $params = [] )
Retrieves metrics about specified crawlers.
- GetCrawlers ( array $params = [] )
Retrieves metadata for all crawlers defined in the customer account.
- GetDataCatalogEncryptionSettings ( array $params = [] )
Retrieves the security configuration for a specified catalog.
- GetDatabase ( array $params = [] )
Retrieves the definition of a specified database.
- GetDatabases ( array $params = [] )
Retrieves all Databases defined in a given Data Catalog.
- GetDataflowGraph ( array $params = [] )
Transforms a Python script into a directed acyclic graph (DAG).
- GetDevEndpoint ( array $params = [] )
Retrieves information about a specified DevEndpoint.
- GetDevEndpoints ( array $params = [] )
Retrieves all the DevEndpoints in this AWS account.
- GetJob ( array $params = [] )
Retrieves an existing job definition.
- GetJobRun ( array $params = [] )
Retrieves the metadata for a given job run.
- GetJobRuns ( array $params = [] )
Retrieves metadata for all runs of a given job definition.
- GetJobs ( array $params = [] )
Retrieves all current job definitions.
- GetMapping ( array $params = [] )
Creates mappings.
- GetPartition ( array $params = [] )
Retrieves information about a specified partition.
- GetPartitions ( array $params = [] )
Retrieves information about the partitions in a table.
- GetPlan ( array $params = [] )
Gets code to perform a specified mapping.
- GetResourcePolicy ( array $params = [] )
Retrieves a specified resource policy.
- GetSecurityConfiguration ( array $params = [] )
Retrieves a specified security configuration.
- GetSecurityConfigurations ( array $params = [] )
Retrieves a list of all security configurations.
- GetTable ( array $params = [] )
Retrieves the Table definition in a Data Catalog for a specified table.
- GetTableVersion ( array $params = [] )
Retrieves a specified version of a table.
- GetTableVersions ( array $params = [] )
Retrieves a list of strings that identify available versions of a specified table.
- GetTables ( array $params = [] )
Retrieves the definitions of some or all of the tables in a given Database.
- GetTags ( array $params = [] )
Retrieves a list of tags associated with a resource.
- GetTrigger ( array $params = [] )
Retrieves the definition of a trigger.
- GetTriggers ( array $params = [] )
Gets all the triggers associated with a job.
- GetUserDefinedFunction ( array $params = [] )
Retrieves a specified function definition from the Data Catalog.
- GetUserDefinedFunctions ( array $params = [] )
Retrieves a multiple function definitions from the Data Catalog.
- GetWorkflow ( array $params = [] )
Retrieves resource metadata for a workflow.
- GetWorkflowRun ( array $params = [] )
Retrieves the metadata for a given workflow run.
- GetWorkflowRunProperties ( array $params = [] )
Retrieves the workflow run properties which were set during the run.
- GetWorkflowRuns ( array $params = [] )
Retrieves metadata for all runs of a given workflow.
- ImportCatalogToGlue ( array $params = [] )
Imports an existing Athena Data Catalog to AWS Glue
- ListCrawlers ( array $params = [] )
Retrieves the names of all crawler resources in this AWS account, or the resources with the specified tag.
- ListDevEndpoints ( array $params = [] )
Retrieves the names of all DevEndpoint resources in this AWS account, or the resources with the specified tag.
- ListJobs ( array $params = [] )
Retrieves the names of all job resources in this AWS account, or the resources with the specified tag.
- ListTriggers ( array $params = [] )
Retrieves the names of all trigger resources in this AWS account, or the resources with the specified tag.
- ListWorkflows ( array $params = [] )
Lists names of workflows created in the account.
- PutDataCatalogEncryptionSettings ( array $params = [] )
Sets the security configuration for a specified catalog.
- PutResourcePolicy ( array $params = [] )
Sets the Data Catalog resource policy for access control.
- PutWorkflowRunProperties ( array $params = [] )
Puts the specified workflow run properties for the given workflow run.
- ResetJobBookmark ( array $params = [] )
Resets a bookmark entry.
- StartCrawler ( array $params = [] )
Starts a crawl using the specified crawler, regardless of what is scheduled.
- StartCrawlerSchedule ( array $params = [] )
Changes the schedule state of the specified crawler to SCHEDULED, unless the crawler is already running or the schedule state is already SCHEDULED.
- StartJobRun ( array $params = [] )
Starts a job run using a job definition.
- StartTrigger ( array $params = [] )
Starts an existing trigger.
- StartWorkflowRun ( array $params = [] )
Starts a new run of the specified workflow.
- StopCrawler ( array $params = [] )
If the specified crawler is running, stops the crawl.
- StopCrawlerSchedule ( array $params = [] )
Sets the schedule state of the specified crawler to NOT_SCHEDULED, but does not stop the crawler if it is already running.
- StopTrigger ( array $params = [] )
Stops a specified trigger.
- TagResource ( array $params = [] )
Adds tags to a resource.
- UntagResource ( array $params = [] )
Removes tags from a resource.
- UpdateClassifier ( array $params = [] )
Modifies an existing classifier (a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field is present).
- UpdateConnection ( array $params = [] )
Updates a connection definition in the Data Catalog.
- UpdateCrawler ( array $params = [] )
Updates a crawler.
- UpdateCrawlerSchedule ( array $params = [] )
Updates the schedule of a crawler using a cron expression.
- UpdateDatabase ( array $params = [] )
Updates an existing database definition in a Data Catalog.
- UpdateDevEndpoint ( array $params = [] )
Updates a specified DevEndpoint.
- UpdateJob ( array $params = [] )
Updates an existing job definition.
- UpdatePartition ( array $params = [] )
Updates a partition.
- UpdateTable ( array $params = [] )
Updates a metadata table in the Data Catalog.
- UpdateTrigger ( array $params = [] )
Updates a trigger definition.
- UpdateUserDefinedFunction ( array $params = [] )
Updates an existing function definition in the Data Catalog.
- UpdateWorkflow ( array $params = [] )
Updates an existing workflow.
Paginators
Paginators handle automatically iterating over paginated API results. Paginators are associated with specific API operations, and they accept the parameters that the corresponding API operation accepts. You can get a paginator from a client class using getPaginator($paginatorName, $operationParameters). This client supports the following paginators:
Operations
BatchCreatePartition
$result = $client->batchCreatePartition([/* ... */]); $promise = $client->batchCreatePartitionAsync([/* ... */]);
Creates one or more partitions in a batch operation.
Parameter Syntax
$result = $client->batchCreatePartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionInputList' => [ // REQUIRED
[
'LastAccessTime' => <integer || string || DateTime>,
'LastAnalyzedTime' => <integer || string || DateTime>,
'Parameters' => ['<string>', ...],
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>', // REQUIRED
'SortOrder' => <integer>, // REQUIRED
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'Values' => ['<string>', ...],
],
// ...
],
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the catalog in which the partion is to be created. Currently, this should be the AWS account ID.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the metadata database in which the partition is to be created.
- PartitionInputList
-
- Required: Yes
- Type: Array of PartitionInput structures
A list of
PartitionInputstructures that define the partitions to be created. - TableName
-
- Required: Yes
- Type: string
The name of the metadata table in which the partition is to be created.
Result Syntax
[
'Errors' => [
[
'ErrorDetail' => [
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
],
'PartitionValues' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- Errors
-
- Type: Array of PartitionError structures
Errors encountered when trying to create the requested partitions.
Errors
-
The input provided was not valid.
-
A resource to be created or added already exists.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An internal service error occurred.
-
A specified entity does not exist
-
The operation timed out.
-
An encryption operation failed.
BatchDeleteConnection
$result = $client->batchDeleteConnection([/* ... */]); $promise = $client->batchDeleteConnectionAsync([/* ... */]);
Deletes a list of connection definitions from the Data Catalog.
Parameter Syntax
$result = $client->batchDeleteConnection([
'CatalogId' => '<string>',
'ConnectionNameList' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Errors' => [
'<NameString>' => [
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
],
// ...
],
'Succeeded' => ['<string>', ...],
]
Result Details
Members
- Errors
-
- Type: Associative array of custom strings keys (NameString) to ErrorDetail structures
A map of the names of connections that were not successfully deleted to error details.
- Succeeded
-
- Type: Array of strings
A list of names of the connection definitions that were successfully deleted.
Errors
-
An internal service error occurred.
-
The operation timed out.
BatchDeletePartition
$result = $client->batchDeletePartition([/* ... */]); $promise = $client->batchDeletePartitionAsync([/* ... */]);
Deletes one or more partitions in a batch operation.
Parameter Syntax
$result = $client->batchDeletePartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionsToDelete' => [ // REQUIRED
[
'Values' => ['<string>', ...], // REQUIRED
],
// ...
],
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the partition to be deleted resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database in which the table in question resides.
- PartitionsToDelete
-
- Required: Yes
- Type: Array of PartitionValueList structures
A list of
PartitionInputstructures that define the partitions to be deleted. - TableName
-
- Required: Yes
- Type: string
The name of the table where the partitions to be deleted is located.
Result Syntax
[
'Errors' => [
[
'ErrorDetail' => [
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
],
'PartitionValues' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- Errors
-
- Type: Array of PartitionError structures
Errors encountered when trying to delete the requested partitions.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
BatchDeleteTable
$result = $client->batchDeleteTable([/* ... */]); $promise = $client->batchDeleteTableAsync([/* ... */]);
Deletes multiple tables at once.
After completing this operation, you will no longer have access to the table versions and partitions that belong to the deleted table. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.
To ensure immediate deletion of all related resources, before calling BatchDeleteTable, use DeleteTableVersion or BatchDeleteTableVersion, and DeletePartition or BatchDeletePartition, to delete any resources that belong to the table.
Parameter Syntax
$result = $client->batchDeleteTable([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TablesToDelete' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the tables to delete reside. For Hive compatibility, this name is entirely lowercase.
- TablesToDelete
-
- Required: Yes
- Type: Array of strings
A list of the table to delete.
Result Syntax
[
'Errors' => [
[
'ErrorDetail' => [
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
],
'TableName' => '<string>',
],
// ...
],
]
Result Details
Members
- Errors
-
- Type: Array of TableError structures
A list of errors encountered in attempting to delete the specified tables.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
BatchDeleteTableVersion
$result = $client->batchDeleteTableVersion([/* ... */]); $promise = $client->batchDeleteTableVersionAsync([/* ... */]);
Deletes a specified batch of versions of a table.
Parameter Syntax
$result = $client->batchDeleteTableVersion([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
'VersionIds' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.
- TableName
-
- Required: Yes
- Type: string
The name of the table. For Hive compatibility, this name is entirely lowercase.
- VersionIds
-
- Required: Yes
- Type: Array of strings
A list of the IDs of versions to be deleted. A
VersionIdis a string representation of an integer. Each version is incremented by 1.
Result Syntax
[
'Errors' => [
[
'ErrorDetail' => [
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
],
'TableName' => '<string>',
'VersionId' => '<string>',
],
// ...
],
]
Result Details
Members
- Errors
-
- Type: Array of TableVersionError structures
A list of errors encountered while trying to delete the specified table versions.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
BatchGetCrawlers
$result = $client->batchGetCrawlers([/* ... */]); $promise = $client->batchGetCrawlersAsync([/* ... */]);
Returns a list of resource metadata for a given list of crawler names. After calling the ListCrawlers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.
Parameter Syntax
$result = $client->batchGetCrawlers([
'CrawlerNames' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Crawlers' => [
[
'Classifiers' => ['<string>', ...],
'Configuration' => '<string>',
'CrawlElapsedTime' => <integer>,
'CrawlerSecurityConfiguration' => '<string>',
'CreationTime' => <DateTime>,
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastCrawl' => [
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'MessagePrefix' => '<string>',
'StartTime' => <DateTime>,
'Status' => 'SUCCEEDED|CANCELLED|FAILED',
],
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Role' => '<string>',
'Schedule' => [
'ScheduleExpression' => '<string>',
'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
],
'SchemaChangePolicy' => [
'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
],
'State' => 'READY|RUNNING|STOPPING',
'TablePrefix' => '<string>',
'Targets' => [
'CatalogTargets' => [
[
'DatabaseName' => '<string>',
'Tables' => ['<string>', ...],
],
// ...
],
'DynamoDBTargets' => [
[
'Path' => '<string>',
],
// ...
],
'JdbcTargets' => [
[
'ConnectionName' => '<string>',
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
'S3Targets' => [
[
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
],
'Version' => <integer>,
],
// ...
],
'CrawlersNotFound' => ['<string>', ...],
]
Result Details
Members
- Crawlers
-
- Type: Array of Crawler structures
A list of crawler definitions.
- CrawlersNotFound
-
- Type: Array of strings
A list of names of crawlers that were not found.
Errors
-
The input provided was not valid.
-
The operation timed out.
BatchGetDevEndpoints
$result = $client->batchGetDevEndpoints([/* ... */]); $promise = $client->batchGetDevEndpointsAsync([/* ... */]);
Returns a list of resource metadata for a given list of DevEndpoint names. After calling the ListDevEndpoints operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.
Parameter Syntax
$result = $client->batchGetDevEndpoints([
'DevEndpointNames' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'DevEndpoints' => [
[
'Arguments' => ['<string>', ...],
'AvailabilityZone' => '<string>',
'CreatedTimestamp' => <DateTime>,
'EndpointName' => '<string>',
'ExtraJarsS3Path' => '<string>',
'ExtraPythonLibsS3Path' => '<string>',
'FailureReason' => '<string>',
'LastModifiedTimestamp' => <DateTime>,
'LastUpdateStatus' => '<string>',
'NumberOfNodes' => <integer>,
'PrivateAddress' => '<string>',
'PublicAddress' => '<string>',
'PublicKey' => '<string>',
'PublicKeys' => ['<string>', ...],
'RoleArn' => '<string>',
'SecurityConfiguration' => '<string>',
'SecurityGroupIds' => ['<string>', ...],
'Status' => '<string>',
'SubnetId' => '<string>',
'VpcId' => '<string>',
'YarnEndpointAddress' => '<string>',
'ZeppelinRemoteSparkInterpreterPort' => <integer>,
],
// ...
],
'DevEndpointsNotFound' => ['<string>', ...],
]
Result Details
Members
- DevEndpoints
-
- Type: Array of DevEndpoint structures
A list of DevEndpoint definitions.
- DevEndpointsNotFound
-
- Type: Array of strings
A list of DevEndpoints not found.
Errors
-
Access to a resource was denied.
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
BatchGetJobs
$result = $client->batchGetJobs([/* ... */]); $promise = $client->batchGetJobsAsync([/* ... */]);
Returns a list of resource metadata for a given list of job names. After calling the ListJobs operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.
Parameter Syntax
$result = $client->batchGetJobs([
'JobNames' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Jobs' => [
[
'AllocatedCapacity' => <integer>,
'Command' => [
'Name' => '<string>',
'PythonVersion' => '<string>',
'ScriptLocation' => '<string>',
],
'Connections' => [
'Connections' => ['<string>', ...],
],
'CreatedOn' => <DateTime>,
'DefaultArguments' => ['<string>', ...],
'Description' => '<string>',
'ExecutionProperty' => [
'MaxConcurrentRuns' => <integer>,
],
'LastModifiedOn' => <DateTime>,
'LogUri' => '<string>',
'MaxCapacity' => <float>,
'MaxRetries' => <integer>,
'Name' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'Role' => '<string>',
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
'JobsNotFound' => ['<string>', ...],
]
Result Details
Members
- Jobs
-
- Type: Array of Job structures
A list of job definitions.
- JobsNotFound
-
- Type: Array of strings
A list of names of jobs not found.
Errors
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
BatchGetPartition
$result = $client->batchGetPartition([/* ... */]); $promise = $client->batchGetPartitionAsync([/* ... */]);
Retrieves partitions in a batch request.
Parameter Syntax
$result = $client->batchGetPartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionsToGet' => [ // REQUIRED
[
'Values' => ['<string>', ...], // REQUIRED
],
// ...
],
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the partitions reside.
- PartitionsToGet
-
- Required: Yes
- Type: Array of PartitionValueList structures
A list of partition values identifying the partitions to retrieve.
- TableName
-
- Required: Yes
- Type: string
The name of the partitions' table.
Result Syntax
[
'Partitions' => [
[
'CreationTime' => <DateTime>,
'DatabaseName' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Parameters' => ['<string>', ...],
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableName' => '<string>',
'Values' => ['<string>', ...],
],
// ...
],
'UnprocessedKeys' => [
[
'Values' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- Partitions
-
- Type: Array of Partition structures
A list of the requested partitions.
- UnprocessedKeys
-
- Type: Array of PartitionValueList structures
A list of the partition values in the request for which partions were not returned.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
The operation timed out.
-
An internal service error occurred.
-
An encryption operation failed.
BatchGetTriggers
$result = $client->batchGetTriggers([/* ... */]); $promise = $client->batchGetTriggersAsync([/* ... */]);
Returns a list of resource metadata for a given list of trigger names. After calling the ListTriggers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.
Parameter Syntax
$result = $client->batchGetTriggers([
'TriggerNames' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Triggers' => [
[
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
// ...
],
'TriggersNotFound' => ['<string>', ...],
]
Result Details
Members
- Triggers
-
- Type: Array of Trigger structures
A list of trigger definitions.
- TriggersNotFound
-
- Type: Array of strings
A list of names of triggers not found.
Errors
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
BatchGetWorkflows
$result = $client->batchGetWorkflows([/* ... */]); $promise = $client->batchGetWorkflowsAsync([/* ... */]);
Returns a list of resource metadata for a given list of workflow names. After calling the ListWorkflows operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.
Parameter Syntax
$result = $client->batchGetWorkflows([
'IncludeGraph' => true || false,
'Names' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'MissingWorkflows' => ['<string>', ...],
'Workflows' => [
[
'CreatedOn' => <DateTime>,
'DefaultRunProperties' => ['<string>', ...],
'Description' => '<string>',
'Graph' => [
'Edges' => [
[
'DestinationId' => '<string>',
'SourceId' => '<string>',
],
// ...
],
'Nodes' => [
[
'CrawlerDetails' => [
'Crawls' => [
[
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
],
// ...
],
],
'JobDetails' => [
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
],
'Name' => '<string>',
'TriggerDetails' => [
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
],
'Type' => 'CRAWLER|JOB|TRIGGER',
'UniqueId' => '<string>',
],
// ...
],
],
'LastModifiedOn' => <DateTime>,
'LastRun' => [
'CompletedOn' => <DateTime>,
'Graph' => [
'Edges' => [
[
'DestinationId' => '<string>',
'SourceId' => '<string>',
],
// ...
],
'Nodes' => [
[
'CrawlerDetails' => [
'Crawls' => [
[
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
],
// ...
],
],
'JobDetails' => [
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
],
'Name' => '<string>',
'TriggerDetails' => [
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
],
'Type' => 'CRAWLER|JOB|TRIGGER',
'UniqueId' => '<string>',
],
// ...
],
],
'Name' => '<string>',
'StartedOn' => <DateTime>,
'Statistics' => [
'FailedActions' => <integer>,
'RunningActions' => <integer>,
'StoppedActions' => <integer>,
'SucceededActions' => <integer>,
'TimeoutActions' => <integer>,
'TotalActions' => <integer>,
],
'Status' => 'RUNNING|COMPLETED',
'WorkflowRunId' => '<string>',
'WorkflowRunProperties' => ['<string>', ...],
],
'Name' => '<string>',
],
// ...
],
]
Result Details
Members
- MissingWorkflows
-
- Type: Array of strings
A list of names of workflows not found.
- Workflows
-
- Type: Array of Workflow structures
A list of workflow resource metadata.
Errors
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
BatchStopJobRun
$result = $client->batchStopJobRun([/* ... */]); $promise = $client->batchStopJobRunAsync([/* ... */]);
Stops one or more job runs for a specified job definition.
Parameter Syntax
$result = $client->batchStopJobRun([
'JobName' => '<string>', // REQUIRED
'JobRunIds' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Errors' => [
[
'ErrorDetail' => [
'ErrorCode' => '<string>',
'ErrorMessage' => '<string>',
],
'JobName' => '<string>',
'JobRunId' => '<string>',
],
// ...
],
'SuccessfulSubmissions' => [
[
'JobName' => '<string>',
'JobRunId' => '<string>',
],
// ...
],
]
Result Details
Members
- Errors
-
- Type: Array of BatchStopJobRunError structures
A list of the errors that were encountered in trying to stop
JobRuns, including theJobRunIdfor which each error was encountered and details about the error. - SuccessfulSubmissions
-
- Type: Array of BatchStopJobRunSuccessfulSubmission structures
A list of the JobRuns that were successfully submitted for stopping.
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
CreateClassifier
$result = $client->createClassifier([/* ... */]); $promise = $client->createClassifierAsync([/* ... */]);
Creates a classifier in the user's account. This can be a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field of the request is present.
Parameter Syntax
$result = $client->createClassifier([
'CsvClassifier' => [
'AllowSingleColumn' => true || false,
'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
'Delimiter' => '<string>',
'DisableValueTrimming' => true || false,
'Header' => ['<string>', ...],
'Name' => '<string>', // REQUIRED
'QuoteSymbol' => '<string>',
],
'GrokClassifier' => [
'Classification' => '<string>', // REQUIRED
'CustomPatterns' => '<string>',
'GrokPattern' => '<string>', // REQUIRED
'Name' => '<string>', // REQUIRED
],
'JsonClassifier' => [
'JsonPath' => '<string>', // REQUIRED
'Name' => '<string>', // REQUIRED
],
'XMLClassifier' => [
'Classification' => '<string>', // REQUIRED
'Name' => '<string>', // REQUIRED
'RowTag' => '<string>',
],
]);
Parameter Details
Members
- CsvClassifier
-
- Type: CreateCsvClassifierRequest structure
A
CsvClassifierobject specifying the classifier to create. - GrokClassifier
-
- Type: CreateGrokClassifierRequest structure
A
GrokClassifierobject specifying the classifier to create. - JsonClassifier
-
- Type: CreateJsonClassifierRequest structure
A
JsonClassifierobject specifying the classifier to create. - XMLClassifier
-
- Type: CreateXMLClassifierRequest structure
An
XMLClassifierobject specifying the classifier to create.
Result Syntax
[]
Result Details
Errors
-
A resource to be created or added already exists.
-
The input provided was not valid.
-
The operation timed out.
CreateConnection
$result = $client->createConnection([/* ... */]); $promise = $client->createConnectionAsync([/* ... */]);
Creates a connection definition in the Data Catalog.
Parameter Syntax
$result = $client->createConnection([
'CatalogId' => '<string>',
'ConnectionInput' => [ // REQUIRED
'ConnectionProperties' => ['<string>', ...], // REQUIRED
'ConnectionType' => 'JDBC|SFTP', // REQUIRED
'Description' => '<string>',
'MatchCriteria' => ['<string>', ...],
'Name' => '<string>', // REQUIRED
'PhysicalConnectionRequirements' => [
'AvailabilityZone' => '<string>',
'SecurityGroupIdList' => ['<string>', ...],
'SubnetId' => '<string>',
],
],
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which to create the connection. If none is provided, the AWS account ID is used by default.
- ConnectionInput
-
- Required: Yes
- Type: ConnectionInput structure
A
ConnectionInputobject defining the connection to create.
Result Syntax
[]
Result Details
Errors
-
A resource to be created or added already exists.
-
The input provided was not valid.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An encryption operation failed.
CreateCrawler
$result = $client->createCrawler([/* ... */]); $promise = $client->createCrawlerAsync([/* ... */]);
Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in the s3Targets field, the jdbcTargets field, or the DynamoDBTargets field.
Parameter Syntax
$result = $client->createCrawler([
'Classifiers' => ['<string>', ...],
'Configuration' => '<string>',
'CrawlerSecurityConfiguration' => '<string>',
'DatabaseName' => '<string>',
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
'Role' => '<string>', // REQUIRED
'Schedule' => '<string>',
'SchemaChangePolicy' => [
'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
],
'TablePrefix' => '<string>',
'Tags' => ['<string>', ...],
'Targets' => [ // REQUIRED
'CatalogTargets' => [
[
'DatabaseName' => '<string>', // REQUIRED
'Tables' => ['<string>', ...], // REQUIRED
],
// ...
],
'DynamoDBTargets' => [
[
'Path' => '<string>',
],
// ...
],
'JdbcTargets' => [
[
'ConnectionName' => '<string>',
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
'S3Targets' => [
[
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
],
]);
Parameter Details
Members
- Classifiers
-
- Type: Array of strings
A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.
- Configuration
-
- Type: string
The crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.
- CrawlerSecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used by this crawler. - DatabaseName
-
- Type: string
The AWS Glue database where results are written, such as:
arn:aws:daylight:us-east-1::database/sometable/*. - Description
-
- Type: string
A description of the new crawler.
- Name
-
- Required: Yes
- Type: string
Name of the new crawler.
- Role
-
- Required: Yes
- Type: string
The IAM role or Amazon Resource Name (ARN) of an IAM role used by the new crawler to access customer resources.
- Schedule
-
- Type: string
A
cronexpression used to specify the schedule. For more information, see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, specifycron(15 12 * * ? *). - SchemaChangePolicy
-
- Type: SchemaChangePolicy structure
The policy for the crawler's update and deletion behavior.
- TablePrefix
-
- Type: string
The table prefix used for catalog tables that are created.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
The tags to use with this crawler request. You can use tags to limit access to the crawler. For more information, see AWS Tags in AWS Glue.
- Targets
-
- Required: Yes
- Type: CrawlerTargets structure
A list of collection of targets to crawl.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
A resource to be created or added already exists.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
CreateDatabase
$result = $client->createDatabase([/* ... */]); $promise = $client->createDatabaseAsync([/* ... */]);
Creates a new database in a Data Catalog.
Parameter Syntax
$result = $client->createDatabase([
'CatalogId' => '<string>',
'DatabaseInput' => [ // REQUIRED
'Description' => '<string>',
'LocationUri' => '<string>',
'Name' => '<string>', // REQUIRED
'Parameters' => ['<string>', ...],
],
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which to create the database. If none is supplied, the AWS account ID is used by default.
- DatabaseInput
-
- Required: Yes
- Type: DatabaseInput structure
A
DatabaseInputobject defining the metadata database to create in the catalog.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
A resource to be created or added already exists.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
CreateDevEndpoint
$result = $client->createDevEndpoint([/* ... */]); $promise = $client->createDevEndpointAsync([/* ... */]);
Creates a new DevEndpoint.
Parameter Syntax
$result = $client->createDevEndpoint([
'Arguments' => ['<string>', ...],
'EndpointName' => '<string>', // REQUIRED
'ExtraJarsS3Path' => '<string>',
'ExtraPythonLibsS3Path' => '<string>',
'NumberOfNodes' => <integer>,
'PublicKey' => '<string>',
'PublicKeys' => ['<string>', ...],
'RoleArn' => '<string>', // REQUIRED
'SecurityConfiguration' => '<string>',
'SecurityGroupIds' => ['<string>', ...],
'SubnetId' => '<string>',
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- Arguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
A map of arguments used to configure the DevEndpoint.
- EndpointName
-
- Required: Yes
- Type: string
The name to be assigned to the new DevEndpoint.
- ExtraJarsS3Path
-
- Type: string
Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.
- ExtraPythonLibsS3Path
-
- Type: string
Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.
Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.
- NumberOfNodes
-
- Type: int
The number of AWS Glue Data Processing Units (DPUs) to allocate to this DevEndpoint.
- PublicKey
-
- Type: string
The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility, as the recommended attribute to use is public keys.
- PublicKeys
-
- Type: Array of strings
A list of public keys to be used by the DevEndpoints for authentication. The use of this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.
If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys: call the
UpdateDevEndpointAPI with the public key content in thedeletePublicKeysattribute, and the list of new keys in theaddPublicKeysattribute. - RoleArn
-
- Required: Yes
- Type: string
The IAM role for the DevEndpoint.
- SecurityConfiguration
-
- Type: string
The name of the SecurityConfiguration structure to be used with this DevEndpoint.
- SecurityGroupIds
-
- Type: Array of strings
Security group IDs for the security groups to be used by the new DevEndpoint.
- SubnetId
-
- Type: string
The subnet ID for the new DevEndpoint to use.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
The tags to use with this DevEndpoint. You may use tags to limit access to the DevEndpoint. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.
Result Syntax
[
'Arguments' => ['<string>', ...],
'AvailabilityZone' => '<string>',
'CreatedTimestamp' => <DateTime>,
'EndpointName' => '<string>',
'ExtraJarsS3Path' => '<string>',
'ExtraPythonLibsS3Path' => '<string>',
'FailureReason' => '<string>',
'NumberOfNodes' => <integer>,
'RoleArn' => '<string>',
'SecurityConfiguration' => '<string>',
'SecurityGroupIds' => ['<string>', ...],
'Status' => '<string>',
'SubnetId' => '<string>',
'VpcId' => '<string>',
'YarnEndpointAddress' => '<string>',
'ZeppelinRemoteSparkInterpreterPort' => <integer>,
]
Result Details
Members
- Arguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The map of arguments used to configure this DevEndpoint.
- AvailabilityZone
-
- Type: string
The AWS availability zone where this DevEndpoint is located.
- CreatedTimestamp
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The point in time at which this DevEndpoint was created.
- EndpointName
-
- Type: string
The name assigned to the new DevEndpoint.
- ExtraJarsS3Path
-
- Type: string
Path to one or more Java Jars in an S3 bucket that will be loaded in your DevEndpoint.
- ExtraPythonLibsS3Path
-
- Type: string
Path(s) to one or more Python libraries in an S3 bucket that will be loaded in your DevEndpoint.
- FailureReason
-
- Type: string
The reason for a current failure in this DevEndpoint.
- NumberOfNodes
-
- Type: int
The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint.
- RoleArn
-
- Type: string
The AWS ARN of the role assigned to the new DevEndpoint.
- SecurityConfiguration
-
- Type: string
The name of the SecurityConfiguration structure being used with this DevEndpoint.
- SecurityGroupIds
-
- Type: Array of strings
The security groups assigned to the new DevEndpoint.
- Status
-
- Type: string
The current status of the new DevEndpoint.
- SubnetId
-
- Type: string
The subnet ID assigned to the new DevEndpoint.
- VpcId
-
- Type: string
The ID of the VPC used by this DevEndpoint.
- YarnEndpointAddress
-
- Type: string
The address of the YARN endpoint used by this DevEndpoint.
- ZeppelinRemoteSparkInterpreterPort
-
- Type: int
The Apache Zeppelin port for the remote Apache Spark interpreter.
Errors
-
Access to a resource was denied.
-
A resource to be created or added already exists.
-
IdempotentParameterMismatchException:
The same unique identifier was associated with two different records.
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
-
A value could not be validated.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
CreateJob
$result = $client->createJob([/* ... */]); $promise = $client->createJobAsync([/* ... */]);
Creates a new job definition.
Parameter Syntax
$result = $client->createJob([
'AllocatedCapacity' => <integer>,
'Command' => [ // REQUIRED
'Name' => '<string>',
'PythonVersion' => '<string>',
'ScriptLocation' => '<string>',
],
'Connections' => [
'Connections' => ['<string>', ...],
],
'DefaultArguments' => ['<string>', ...],
'Description' => '<string>',
'ExecutionProperty' => [
'MaxConcurrentRuns' => <integer>,
],
'LogUri' => '<string>',
'MaxCapacity' => <float>,
'MaxRetries' => <integer>,
'Name' => '<string>', // REQUIRED
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'Role' => '<string>', // REQUIRED
'SecurityConfiguration' => '<string>',
'Tags' => ['<string>', ...],
'Timeout' => <integer>,
'WorkerType' => '<string>',
]);
Parameter Details
Members
- AllocatedCapacity
-
- Type: int
This parameter is deprecated. Use
MaxCapacityinstead.The number of AWS Glue data processing units (DPUs) to allocate to this Job. You can allocate from 2 to 100 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
- Command
-
- Required: Yes
- Type: JobCommand structure
The
JobCommandthat executes this job. - Connections
-
- Type: ConnectionsList structure
The connections used for this job.
- DefaultArguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The default arguments for this job.
You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.
For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.
- Description
-
- Type: string
Description of the job being defined.
- ExecutionProperty
-
- Type: ExecutionProperty structure
An
ExecutionPropertyspecifying the maximum number of concurrent runs allowed for this job. - LogUri
-
- Type: string
This field is reserved for future use.
- MaxCapacity
-
- Type: double
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
Do not set
Max Capacityif usingWorkerTypeandNumberOfWorkers.The value that can be allocated for
MaxCapacitydepends on whether you are running a Python shell job or an Apache Spark ETL job:-
When you specify a Python shell job (
JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU. -
When you specify an Apache Spark ETL job (
JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.
- MaxRetries
-
- Type: int
The maximum number of times to retry this job if it fails.
- Name
-
- Required: Yes
- Type: string
The name you assign to this job definition. It must be unique in your account.
- NotificationProperty
-
- Type: NotificationProperty structure
Specifies configuration properties of a job notification.
- NumberOfWorkers
-
- Type: int
The number of workers of a defined
workerTypethat are allocated when a job runs.The maximum number of workers you can define are 299 for
G.1X, and 149 forG.2X. - Role
-
- Required: Yes
- Type: string
The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
- SecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used with this job. - Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
The tags to use with this job. You may use tags to limit access to the job. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.
- Timeout
-
- Type: int
The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters
TIMEOUTstatus. The default is 2,880 minutes (48 hours). - WorkerType
-
- Type: string
The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.
-
For the
Standardworker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker. -
For the
G.1Xworker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs. -
For the
G.2Xworker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
IdempotentParameterMismatchException:
The same unique identifier was associated with two different records.
-
A resource to be created or added already exists.
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
CreatePartition
$result = $client->createPartition([/* ... */]); $promise = $client->createPartitionAsync([/* ... */]);
Creates a new partition.
Parameter Syntax
$result = $client->createPartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionInput' => [ // REQUIRED
'LastAccessTime' => <integer || string || DateTime>,
'LastAnalyzedTime' => <integer || string || DateTime>,
'Parameters' => ['<string>', ...],
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>', // REQUIRED
'SortOrder' => <integer>, // REQUIRED
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'Values' => ['<string>', ...],
],
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the catalog in which the partion is to be created. Currently, this should be the AWS account ID.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the metadata database in which the partition is to be created.
- PartitionInput
-
- Required: Yes
- Type: PartitionInput structure
A
PartitionInputstructure defining the partition to be created. - TableName
-
- Required: Yes
- Type: string
The name of the metadata table in which the partition is to be created.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
A resource to be created or added already exists.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An internal service error occurred.
-
A specified entity does not exist
-
The operation timed out.
-
An encryption operation failed.
CreateScript
$result = $client->createScript([/* ... */]); $promise = $client->createScriptAsync([/* ... */]);
Transforms a directed acyclic graph (DAG) into code.
Parameter Syntax
$result = $client->createScript([
'DagEdges' => [
[
'Source' => '<string>', // REQUIRED
'Target' => '<string>', // REQUIRED
'TargetParameter' => '<string>',
],
// ...
],
'DagNodes' => [
[
'Args' => [ // REQUIRED
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
'Id' => '<string>', // REQUIRED
'LineNumber' => <integer>,
'NodeType' => '<string>', // REQUIRED
],
// ...
],
'Language' => 'PYTHON|SCALA',
]);
Parameter Details
Members
- DagEdges
-
- Type: Array of CodeGenEdge structures
A list of the edges in the DAG.
- DagNodes
-
- Type: Array of CodeGenNode structures
A list of the nodes in the DAG.
- Language
-
- Type: string
The programming language of the resulting code from the DAG.
Result Syntax
[
'PythonScript' => '<string>',
'ScalaCode' => '<string>',
]
Result Details
Members
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
CreateSecurityConfiguration
$result = $client->createSecurityConfiguration([/* ... */]); $promise = $client->createSecurityConfigurationAsync([/* ... */]);
Creates a new security configuration.
Parameter Syntax
$result = $client->createSecurityConfiguration([
'EncryptionConfiguration' => [ // REQUIRED
'CloudWatchEncryption' => [
'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
'KmsKeyArn' => '<string>',
],
'JobBookmarksEncryption' => [
'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
'KmsKeyArn' => '<string>',
],
'S3Encryption' => [
[
'KmsKeyArn' => '<string>',
'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
],
// ...
],
],
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- EncryptionConfiguration
-
- Required: Yes
- Type: EncryptionConfiguration structure
The encryption configuration for the new security configuration.
- Name
-
- Required: Yes
- Type: string
The name for the new security configuration.
Result Syntax
[
'CreatedTimestamp' => <DateTime>,
'Name' => '<string>',
]
Result Details
Members
Errors
-
A resource to be created or added already exists.
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
CreateTable
$result = $client->createTable([/* ... */]); $promise = $client->createTableAsync([/* ... */]);
Creates a new table definition in the Data Catalog.
Parameter Syntax
$result = $client->createTable([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TableInput' => [ // REQUIRED
'Description' => '<string>',
'LastAccessTime' => <integer || string || DateTime>,
'LastAnalyzedTime' => <integer || string || DateTime>,
'Name' => '<string>', // REQUIRED
'Owner' => '<string>',
'Parameters' => ['<string>', ...],
'PartitionKeys' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Retention' => <integer>,
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>', // REQUIRED
'SortOrder' => <integer>, // REQUIRED
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableType' => '<string>',
'ViewExpandedText' => '<string>',
'ViewOriginalText' => '<string>',
],
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which to create the
Table. If none is supplied, the AWS account ID is used by default. - DatabaseName
-
- Required: Yes
- Type: string
The catalog database in which to create the new table. For Hive compatibility, this name is entirely lowercase.
- TableInput
-
- Required: Yes
- Type: TableInput structure
The
TableInputobject that defines the metadata table to create in the catalog.
Result Syntax
[]
Result Details
Errors
-
A resource to be created or added already exists.
-
The input provided was not valid.
-
A specified entity does not exist
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
CreateTrigger
$result = $client->createTrigger([/* ... */]); $promise = $client->createTriggerAsync([/* ... */]);
Creates a new trigger.
Parameter Syntax
$result = $client->createTrigger([
'Actions' => [ // REQUIRED
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'StartOnCreation' => true || false,
'Tags' => ['<string>', ...],
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND', // REQUIRED
'WorkflowName' => '<string>',
]);
Parameter Details
Members
- Actions
-
- Required: Yes
- Type: Array of Action structures
The actions initiated by this trigger when it fires.
- Description
-
- Type: string
A description of the new trigger.
- Name
-
- Required: Yes
- Type: string
The name of the trigger.
- Predicate
-
- Type: Predicate structure
A predicate to specify when the new trigger should fire.
This field is required when the trigger type is
CONDITIONAL. - Schedule
-
- Type: string
A
cronexpression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify:cron(15 12 * * ? *).This field is required when the trigger type is SCHEDULED.
- StartOnCreation
-
- Type: boolean
Set to
trueto startSCHEDULEDandCONDITIONALtriggers when created. True is not supported forON_DEMANDtriggers. - Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
The tags to use with this trigger. You may use tags to limit access to the trigger. For more information about tags in AWS Glue, see AWS Tags in AWS Glue in the developer guide.
- Type
-
- Required: Yes
- Type: string
The type of the new trigger.
- WorkflowName
-
- Type: string
The name of the workflow associated with the trigger.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
A resource to be created or added already exists.
-
A specified entity does not exist
-
The input provided was not valid.
-
IdempotentParameterMismatchException:
The same unique identifier was associated with two different records.
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
CreateUserDefinedFunction
$result = $client->createUserDefinedFunction([/* ... */]); $promise = $client->createUserDefinedFunctionAsync([/* ... */]);
Creates a new function definition in the Data Catalog.
Parameter Syntax
$result = $client->createUserDefinedFunction([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'FunctionInput' => [ // REQUIRED
'ClassName' => '<string>',
'FunctionName' => '<string>',
'OwnerName' => '<string>',
'OwnerType' => 'USER|ROLE|GROUP',
'ResourceUris' => [
[
'ResourceType' => 'JAR|FILE|ARCHIVE',
'Uri' => '<string>',
],
// ...
],
],
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which to create the function. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database in which to create the function.
- FunctionInput
-
- Required: Yes
- Type: UserDefinedFunctionInput structure
A
FunctionInputobject that defines the function to create in the Data Catalog.
Result Syntax
[]
Result Details
Errors
-
A resource to be created or added already exists.
-
The input provided was not valid.
-
An internal service error occurred.
-
A specified entity does not exist
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An encryption operation failed.
CreateWorkflow
$result = $client->createWorkflow([/* ... */]); $promise = $client->createWorkflowAsync([/* ... */]);
Creates a new workflow.
Parameter Syntax
$result = $client->createWorkflow([
'DefaultRunProperties' => ['<string>', ...],
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- DefaultRunProperties
-
- Type: Associative array of custom strings keys (IdString) to strings
A collection of properties to be used as part of each execution of the workflow.
- Description
-
- Type: string
A description of the workflow.
- Name
-
- Required: Yes
- Type: string
The name to be assigned to the workflow. It should be unique within your account.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
The tags to be used with this workflow.
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
A resource to be created or added already exists.
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
DeleteClassifier
$result = $client->deleteClassifier([/* ... */]); $promise = $client->deleteClassifierAsync([/* ... */]);
Removes a classifier from the Data Catalog.
Parameter Syntax
$result = $client->deleteClassifier([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The operation timed out.
DeleteConnection
$result = $client->deleteConnection([/* ... */]); $promise = $client->deleteConnectionAsync([/* ... */]);
Deletes a connection from the Data Catalog.
Parameter Syntax
$result = $client->deleteConnection([
'CatalogId' => '<string>',
'ConnectionName' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The operation timed out.
DeleteCrawler
$result = $client->deleteCrawler([/* ... */]); $promise = $client->deleteCrawlerAsync([/* ... */]);
Removes a specified crawler from the AWS Glue Data Catalog, unless the crawler state is RUNNING.
Parameter Syntax
$result = $client->deleteCrawler([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The operation cannot be performed because the crawler is already running.
-
SchedulerTransitioningException:
The specified scheduler is transitioning.
-
The operation timed out.
DeleteDatabase
$result = $client->deleteDatabase([/* ... */]); $promise = $client->deleteDatabaseAsync([/* ... */]);
Removes a specified Database from a Data Catalog.
After completing this operation, you will no longer have access to the tables (and all table versions and partitions that might belong to the tables) and the user-defined functions in the deleted database. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.
To ensure immediate deletion of all related resources, before calling DeleteDatabase, use DeleteTableVersion or BatchDeleteTableVersion, DeletePartition or BatchDeletePartition, DeleteUserDefinedFunction, and DeleteTable or BatchDeleteTable, to delete any resources that belong to the database.
Parameter Syntax
$result = $client->deleteDatabase([
'CatalogId' => '<string>',
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeleteDevEndpoint
$result = $client->deleteDevEndpoint([/* ... */]); $promise = $client->deleteDevEndpointAsync([/* ... */]);
Deletes a specified DevEndpoint.
Parameter Syntax
$result = $client->deleteDevEndpoint([
'EndpointName' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
DeleteJob
$result = $client->deleteJob([/* ... */]); $promise = $client->deleteJobAsync([/* ... */]);
Deletes a specified job definition. If the job definition is not found, no exception is thrown.
Parameter Syntax
$result = $client->deleteJob([
'JobName' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'JobName' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeletePartition
$result = $client->deletePartition([/* ... */]); $promise = $client->deletePartitionAsync([/* ... */]);
Deletes a specified partition.
Parameter Syntax
$result = $client->deletePartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionValues' => ['<string>', ...], // REQUIRED
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the partition to be deleted resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database in which the table in question resides.
- PartitionValues
-
- Required: Yes
- Type: Array of strings
The values that define the partition.
- TableName
-
- Required: Yes
- Type: string
The name of the table where the partition to be deleted is located.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeleteResourcePolicy
$result = $client->deleteResourcePolicy([/* ... */]); $promise = $client->deleteResourcePolicyAsync([/* ... */]);
Deletes a specified policy.
Parameter Syntax
$result = $client->deleteResourcePolicy([
'PolicyHashCondition' => '<string>',
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
-
ConditionCheckFailureException:
A specified condition was not satisfied.
DeleteSecurityConfiguration
$result = $client->deleteSecurityConfiguration([/* ... */]); $promise = $client->deleteSecurityConfigurationAsync([/* ... */]);
Deletes a specified security configuration.
Parameter Syntax
$result = $client->deleteSecurityConfiguration([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeleteTable
$result = $client->deleteTable([/* ... */]); $promise = $client->deleteTableAsync([/* ... */]);
Removes a table definition from the Data Catalog.
After completing this operation, you will no longer have access to the table versions and partitions that belong to the deleted table. AWS Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.
To ensure immediate deletion of all related resources, before calling DeleteTable, use DeleteTableVersion or BatchDeleteTableVersion, and DeletePartition or BatchDeletePartition, to delete any resources that belong to the table.
Parameter Syntax
$result = $client->deleteTable([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database in which the table resides. For Hive compatibility, this name is entirely lowercase.
- Name
-
- Required: Yes
- Type: string
The name of the table to be deleted. For Hive compatibility, this name is entirely lowercase.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeleteTableVersion
$result = $client->deleteTableVersion([/* ... */]); $promise = $client->deleteTableVersionAsync([/* ... */]);
Deletes a specified version of a table.
Parameter Syntax
$result = $client->deleteTableVersion([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
'VersionId' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.
- TableName
-
- Required: Yes
- Type: string
The name of the table. For Hive compatibility, this name is entirely lowercase.
- VersionId
-
- Required: Yes
- Type: string
The ID of the table version to be deleted. A
VersionIDis a string representation of an integer. Each version is incremented by 1.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeleteTrigger
$result = $client->deleteTrigger([/* ... */]); $promise = $client->deleteTriggerAsync([/* ... */]);
Deletes a specified trigger. If the trigger is not found, no exception is thrown.
Parameter Syntax
$result = $client->deleteTrigger([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
DeleteUserDefinedFunction
$result = $client->deleteUserDefinedFunction([/* ... */]); $promise = $client->deleteUserDefinedFunctionAsync([/* ... */]);
Deletes an existing function definition from the Data Catalog.
Parameter Syntax
$result = $client->deleteUserDefinedFunction([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'FunctionName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the function to be deleted is located. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the function is located.
- FunctionName
-
- Required: Yes
- Type: string
The name of the function definition to be deleted.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
DeleteWorkflow
$result = $client->deleteWorkflow([/* ... */]); $promise = $client->deleteWorkflowAsync([/* ... */]);
Deletes a workflow.
Parameter Syntax
$result = $client->deleteWorkflow([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
GetCatalogImportStatus
$result = $client->getCatalogImportStatus([/* ... */]); $promise = $client->getCatalogImportStatusAsync([/* ... */]);
Retrieves the status of a migration operation.
Parameter Syntax
$result = $client->getCatalogImportStatus([
'CatalogId' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'ImportStatus' => [
'ImportCompleted' => true || false,
'ImportTime' => <DateTime>,
'ImportedBy' => '<string>',
],
]
Result Details
Members
- ImportStatus
-
- Type: CatalogImportStatus structure
The status of the specified catalog migration.
Errors
-
An internal service error occurred.
-
The operation timed out.
GetClassifier
$result = $client->getClassifier([/* ... */]); $promise = $client->getClassifierAsync([/* ... */]);
Retrieve a classifier by name.
Parameter Syntax
$result = $client->getClassifier([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Classifier' => [
'CsvClassifier' => [
'AllowSingleColumn' => true || false,
'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
'CreationTime' => <DateTime>,
'Delimiter' => '<string>',
'DisableValueTrimming' => true || false,
'Header' => ['<string>', ...],
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'QuoteSymbol' => '<string>',
'Version' => <integer>,
],
'GrokClassifier' => [
'Classification' => '<string>',
'CreationTime' => <DateTime>,
'CustomPatterns' => '<string>',
'GrokPattern' => '<string>',
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Version' => <integer>,
],
'JsonClassifier' => [
'CreationTime' => <DateTime>,
'JsonPath' => '<string>',
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Version' => <integer>,
],
'XMLClassifier' => [
'Classification' => '<string>',
'CreationTime' => <DateTime>,
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'RowTag' => '<string>',
'Version' => <integer>,
],
],
]
Result Details
Members
- Classifier
-
- Type: Classifier structure
The requested classifier.
Errors
-
A specified entity does not exist
-
The operation timed out.
GetClassifiers
$result = $client->getClassifiers([/* ... */]); $promise = $client->getClassifiersAsync([/* ... */]);
Lists all classifier objects in the Data Catalog.
Parameter Syntax
$result = $client->getClassifiers([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'Classifiers' => [
[
'CsvClassifier' => [
'AllowSingleColumn' => true || false,
'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
'CreationTime' => <DateTime>,
'Delimiter' => '<string>',
'DisableValueTrimming' => true || false,
'Header' => ['<string>', ...],
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'QuoteSymbol' => '<string>',
'Version' => <integer>,
],
'GrokClassifier' => [
'Classification' => '<string>',
'CreationTime' => <DateTime>,
'CustomPatterns' => '<string>',
'GrokPattern' => '<string>',
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Version' => <integer>,
],
'JsonClassifier' => [
'CreationTime' => <DateTime>,
'JsonPath' => '<string>',
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Version' => <integer>,
],
'XMLClassifier' => [
'Classification' => '<string>',
'CreationTime' => <DateTime>,
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'RowTag' => '<string>',
'Version' => <integer>,
],
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- Classifiers
-
- Type: Array of Classifier structures
The requested list of classifier objects.
- NextToken
-
- Type: string
A continuation token.
Errors
-
The operation timed out.
GetConnection
$result = $client->getConnection([/* ... */]); $promise = $client->getConnectionAsync([/* ... */]);
Retrieves a connection definition from the Data Catalog.
Parameter Syntax
$result = $client->getConnection([
'CatalogId' => '<string>',
'HidePassword' => true || false,
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default.
- HidePassword
-
- Type: boolean
Allows you to retrieve the connection metadata without returning the password. For instance, the AWS Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the AWS KMS key to decrypt the password, but does have permission to access the rest of the connection properties.
- Name
-
- Required: Yes
- Type: string
The name of the connection definition to retrieve.
Result Syntax
[
'Connection' => [
'ConnectionProperties' => ['<string>', ...],
'ConnectionType' => 'JDBC|SFTP',
'CreationTime' => <DateTime>,
'Description' => '<string>',
'LastUpdatedBy' => '<string>',
'LastUpdatedTime' => <DateTime>,
'MatchCriteria' => ['<string>', ...],
'Name' => '<string>',
'PhysicalConnectionRequirements' => [
'AvailabilityZone' => '<string>',
'SecurityGroupIdList' => ['<string>', ...],
'SubnetId' => '<string>',
],
],
]
Result Details
Members
- Connection
-
- Type: Connection structure
The requested connection definition.
Errors
-
A specified entity does not exist
-
The operation timed out.
-
The input provided was not valid.
-
An encryption operation failed.
GetConnections
$result = $client->getConnections([/* ... */]); $promise = $client->getConnectionsAsync([/* ... */]);
Retrieves a list of connection definitions from the Data Catalog.
Parameter Syntax
$result = $client->getConnections([
'CatalogId' => '<string>',
'Filter' => [
'ConnectionType' => 'JDBC|SFTP',
'MatchCriteria' => ['<string>', ...],
],
'HidePassword' => true || false,
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which the connections reside. If none is provided, the AWS account ID is used by default.
- Filter
-
- Type: GetConnectionsFilter structure
A filter that controls which connections will be returned.
- HidePassword
-
- Type: boolean
Allows you to retrieve the connection metadata without returning the password. For instance, the AWS Glue console uses this flag to retrieve the connection, and does not display the password. Set this parameter when the caller might not have permission to use the AWS KMS key to decrypt the password, but does have permission to access the rest of the connection properties.
- MaxResults
-
- Type: int
The maximum number of connections to return in one response.
- NextToken
-
- Type: string
A continuation token, if this is a continuation call.
Result Syntax
[
'ConnectionList' => [
[
'ConnectionProperties' => ['<string>', ...],
'ConnectionType' => 'JDBC|SFTP',
'CreationTime' => <DateTime>,
'Description' => '<string>',
'LastUpdatedBy' => '<string>',
'LastUpdatedTime' => <DateTime>,
'MatchCriteria' => ['<string>', ...],
'Name' => '<string>',
'PhysicalConnectionRequirements' => [
'AvailabilityZone' => '<string>',
'SecurityGroupIdList' => ['<string>', ...],
'SubnetId' => '<string>',
],
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- ConnectionList
-
- Type: Array of Connection structures
A list of requested connection definitions.
- NextToken
-
- Type: string
A continuation token, if the list of connections returned does not include the last of the filtered connections.
Errors
-
A specified entity does not exist
-
The operation timed out.
-
The input provided was not valid.
-
An encryption operation failed.
GetCrawler
$result = $client->getCrawler([/* ... */]); $promise = $client->getCrawlerAsync([/* ... */]);
Retrieves metadata for a specified crawler.
Parameter Syntax
$result = $client->getCrawler([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Crawler' => [
'Classifiers' => ['<string>', ...],
'Configuration' => '<string>',
'CrawlElapsedTime' => <integer>,
'CrawlerSecurityConfiguration' => '<string>',
'CreationTime' => <DateTime>,
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastCrawl' => [
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'MessagePrefix' => '<string>',
'StartTime' => <DateTime>,
'Status' => 'SUCCEEDED|CANCELLED|FAILED',
],
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Role' => '<string>',
'Schedule' => [
'ScheduleExpression' => '<string>',
'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
],
'SchemaChangePolicy' => [
'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
],
'State' => 'READY|RUNNING|STOPPING',
'TablePrefix' => '<string>',
'Targets' => [
'CatalogTargets' => [
[
'DatabaseName' => '<string>',
'Tables' => ['<string>', ...],
],
// ...
],
'DynamoDBTargets' => [
[
'Path' => '<string>',
],
// ...
],
'JdbcTargets' => [
[
'ConnectionName' => '<string>',
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
'S3Targets' => [
[
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
],
'Version' => <integer>,
],
]
Result Details
Members
- Crawler
-
- Type: Crawler structure
The metadata for the specified crawler.
Errors
-
A specified entity does not exist
-
The operation timed out.
GetCrawlerMetrics
$result = $client->getCrawlerMetrics([/* ... */]); $promise = $client->getCrawlerMetricsAsync([/* ... */]);
Retrieves metrics about specified crawlers.
Parameter Syntax
$result = $client->getCrawlerMetrics([
'CrawlerNameList' => ['<string>', ...],
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'CrawlerMetricsList' => [
[
'CrawlerName' => '<string>',
'LastRuntimeSeconds' => <float>,
'MedianRuntimeSeconds' => <float>,
'StillEstimating' => true || false,
'TablesCreated' => <integer>,
'TablesDeleted' => <integer>,
'TablesUpdated' => <integer>,
'TimeLeftSeconds' => <float>,
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- CrawlerMetricsList
-
- Type: Array of CrawlerMetrics structures
A list of metrics for the specified crawler.
- NextToken
-
- Type: string
A continuation token, if the returned list does not contain the last metric available.
Errors
-
The operation timed out.
GetCrawlers
$result = $client->getCrawlers([/* ... */]); $promise = $client->getCrawlersAsync([/* ... */]);
Retrieves metadata for all crawlers defined in the customer account.
Parameter Syntax
$result = $client->getCrawlers([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'Crawlers' => [
[
'Classifiers' => ['<string>', ...],
'Configuration' => '<string>',
'CrawlElapsedTime' => <integer>,
'CrawlerSecurityConfiguration' => '<string>',
'CreationTime' => <DateTime>,
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastCrawl' => [
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'MessagePrefix' => '<string>',
'StartTime' => <DateTime>,
'Status' => 'SUCCEEDED|CANCELLED|FAILED',
],
'LastUpdated' => <DateTime>,
'Name' => '<string>',
'Role' => '<string>',
'Schedule' => [
'ScheduleExpression' => '<string>',
'State' => 'SCHEDULED|NOT_SCHEDULED|TRANSITIONING',
],
'SchemaChangePolicy' => [
'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
],
'State' => 'READY|RUNNING|STOPPING',
'TablePrefix' => '<string>',
'Targets' => [
'CatalogTargets' => [
[
'DatabaseName' => '<string>',
'Tables' => ['<string>', ...],
],
// ...
],
'DynamoDBTargets' => [
[
'Path' => '<string>',
],
// ...
],
'JdbcTargets' => [
[
'ConnectionName' => '<string>',
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
'S3Targets' => [
[
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
],
'Version' => <integer>,
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- Crawlers
-
- Type: Array of Crawler structures
A list of crawler metadata.
- NextToken
-
- Type: string
A continuation token, if the returned list has not reached the end of those defined in this customer account.
Errors
-
The operation timed out.
GetDataCatalogEncryptionSettings
$result = $client->getDataCatalogEncryptionSettings([/* ... */]); $promise = $client->getDataCatalogEncryptionSettingsAsync([/* ... */]);
Retrieves the security configuration for a specified catalog.
Parameter Syntax
$result = $client->getDataCatalogEncryptionSettings([
'CatalogId' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'DataCatalogEncryptionSettings' => [
'ConnectionPasswordEncryption' => [
'AwsKmsKeyId' => '<string>',
'ReturnConnectionPasswordEncrypted' => true || false,
],
'EncryptionAtRest' => [
'CatalogEncryptionMode' => 'DISABLED|SSE-KMS',
'SseAwsKmsKeyId' => '<string>',
],
],
]
Result Details
Members
- DataCatalogEncryptionSettings
-
- Type: DataCatalogEncryptionSettings structure
The requested security configuration.
Errors
-
An internal service error occurred.
-
The input provided was not valid.
-
The operation timed out.
GetDatabase
$result = $client->getDatabase([/* ... */]); $promise = $client->getDatabaseAsync([/* ... */]);
Retrieves the definition of a specified database.
Parameter Syntax
$result = $client->getDatabase([
'CatalogId' => '<string>',
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Database' => [
'CreateTime' => <DateTime>,
'Description' => '<string>',
'LocationUri' => '<string>',
'Name' => '<string>',
'Parameters' => ['<string>', ...],
],
]
Result Details
Members
- Database
-
- Type: Database structure
The definition of the specified database in the catalog.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetDatabases
$result = $client->getDatabases([/* ... */]); $promise = $client->getDatabasesAsync([/* ... */]);
Retrieves all Databases defined in a given Data Catalog.
Parameter Syntax
$result = $client->getDatabases([
'CatalogId' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog from which to retrieve
Databases. If none is supplied, the AWS account ID is used by default. - MaxResults
-
- Type: int
The maximum number of databases to return in one response.
- NextToken
-
- Type: string
A continuation token, if this is a continuation call.
Result Syntax
[
'DatabaseList' => [
[
'CreateTime' => <DateTime>,
'Description' => '<string>',
'LocationUri' => '<string>',
'Name' => '<string>',
'Parameters' => ['<string>', ...],
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- DatabaseList
-
- Required: Yes
- Type: Array of Database structures
A list of
Databaseobjects from the specified catalog. - NextToken
-
- Type: string
A continuation token for paginating the returned list of tokens, returned if the current segment of the list is not the last.
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetDataflowGraph
$result = $client->getDataflowGraph([/* ... */]); $promise = $client->getDataflowGraphAsync([/* ... */]);
Transforms a Python script into a directed acyclic graph (DAG).
Parameter Syntax
$result = $client->getDataflowGraph([
'PythonScript' => '<string>',
]);
Parameter Details
Result Syntax
[
'DagEdges' => [
[
'Source' => '<string>',
'Target' => '<string>',
'TargetParameter' => '<string>',
],
// ...
],
'DagNodes' => [
[
'Args' => [
[
'Name' => '<string>',
'Param' => true || false,
'Value' => '<string>',
],
// ...
],
'Id' => '<string>',
'LineNumber' => <integer>,
'NodeType' => '<string>',
],
// ...
],
]
Result Details
Members
- DagEdges
-
- Type: Array of CodeGenEdge structures
A list of the edges in the resulting DAG.
- DagNodes
-
- Type: Array of CodeGenNode structures
A list of the nodes in the resulting DAG.
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
GetDevEndpoint
$result = $client->getDevEndpoint([/* ... */]); $promise = $client->getDevEndpointAsync([/* ... */]);
Retrieves information about a specified DevEndpoint.
When you create a development endpoint in a virtual private cloud (VPC), AWS Glue returns only a private IP address, and the public IP address field is not populated. When you create a non-VPC development endpoint, AWS Glue returns only a public IP address.
Parameter Syntax
$result = $client->getDevEndpoint([
'EndpointName' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'DevEndpoint' => [
'Arguments' => ['<string>', ...],
'AvailabilityZone' => '<string>',
'CreatedTimestamp' => <DateTime>,
'EndpointName' => '<string>',
'ExtraJarsS3Path' => '<string>',
'ExtraPythonLibsS3Path' => '<string>',
'FailureReason' => '<string>',
'LastModifiedTimestamp' => <DateTime>,
'LastUpdateStatus' => '<string>',
'NumberOfNodes' => <integer>,
'PrivateAddress' => '<string>',
'PublicAddress' => '<string>',
'PublicKey' => '<string>',
'PublicKeys' => ['<string>', ...],
'RoleArn' => '<string>',
'SecurityConfiguration' => '<string>',
'SecurityGroupIds' => ['<string>', ...],
'Status' => '<string>',
'SubnetId' => '<string>',
'VpcId' => '<string>',
'YarnEndpointAddress' => '<string>',
'ZeppelinRemoteSparkInterpreterPort' => <integer>,
],
]
Result Details
Members
- DevEndpoint
-
- Type: DevEndpoint structure
A DevEndpoint definition.
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
GetDevEndpoints
$result = $client->getDevEndpoints([/* ... */]); $promise = $client->getDevEndpointsAsync([/* ... */]);
Retrieves all the DevEndpoints in this AWS account.
When you create a development endpoint in a virtual private cloud (VPC), AWS Glue returns only a private IP address and the public IP address field is not populated. When you create a non-VPC development endpoint, AWS Glue returns only a public IP address.
Parameter Syntax
$result = $client->getDevEndpoints([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'DevEndpoints' => [
[
'Arguments' => ['<string>', ...],
'AvailabilityZone' => '<string>',
'CreatedTimestamp' => <DateTime>,
'EndpointName' => '<string>',
'ExtraJarsS3Path' => '<string>',
'ExtraPythonLibsS3Path' => '<string>',
'FailureReason' => '<string>',
'LastModifiedTimestamp' => <DateTime>,
'LastUpdateStatus' => '<string>',
'NumberOfNodes' => <integer>,
'PrivateAddress' => '<string>',
'PublicAddress' => '<string>',
'PublicKey' => '<string>',
'PublicKeys' => ['<string>', ...],
'RoleArn' => '<string>',
'SecurityConfiguration' => '<string>',
'SecurityGroupIds' => ['<string>', ...],
'Status' => '<string>',
'SubnetId' => '<string>',
'VpcId' => '<string>',
'YarnEndpointAddress' => '<string>',
'ZeppelinRemoteSparkInterpreterPort' => <integer>,
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- DevEndpoints
-
- Type: Array of DevEndpoint structures
A list of DevEndpoint definitions.
- NextToken
-
- Type: string
A continuation token, if not all DevEndpoint definitions have yet been returned.
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
GetJob
$result = $client->getJob([/* ... */]); $promise = $client->getJobAsync([/* ... */]);
Retrieves an existing job definition.
Parameter Syntax
$result = $client->getJob([
'JobName' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Job' => [
'AllocatedCapacity' => <integer>,
'Command' => [
'Name' => '<string>',
'PythonVersion' => '<string>',
'ScriptLocation' => '<string>',
],
'Connections' => [
'Connections' => ['<string>', ...],
],
'CreatedOn' => <DateTime>,
'DefaultArguments' => ['<string>', ...],
'Description' => '<string>',
'ExecutionProperty' => [
'MaxConcurrentRuns' => <integer>,
],
'LastModifiedOn' => <DateTime>,
'LogUri' => '<string>',
'MaxCapacity' => <float>,
'MaxRetries' => <integer>,
'Name' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'Role' => '<string>',
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
'WorkerType' => 'Standard|G.1X|G.2X',
],
]
Result Details
Members
- Job
-
- Type: Job structure
The requested job definition.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetJobRun
$result = $client->getJobRun([/* ... */]); $promise = $client->getJobRunAsync([/* ... */]);
Retrieves the metadata for a given job run.
Parameter Syntax
$result = $client->getJobRun([
'JobName' => '<string>', // REQUIRED
'PredecessorsIncluded' => true || false,
'RunId' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'JobRun' => [
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
]
Result Details
Members
- JobRun
-
- Type: JobRun structure
The requested job-run metadata.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetJobRuns
$result = $client->getJobRuns([/* ... */]); $promise = $client->getJobRunsAsync([/* ... */]);
Retrieves metadata for all runs of a given job definition.
Parameter Syntax
$result = $client->getJobRuns([
'JobName' => '<string>', // REQUIRED
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- JobRuns
-
- Type: Array of JobRun structures
A list of job-run metadata objects.
- NextToken
-
- Type: string
A continuation token, if not all requested job runs have been returned.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetJobs
$result = $client->getJobs([/* ... */]); $promise = $client->getJobsAsync([/* ... */]);
Retrieves all current job definitions.
Parameter Syntax
$result = $client->getJobs([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'Jobs' => [
[
'AllocatedCapacity' => <integer>,
'Command' => [
'Name' => '<string>',
'PythonVersion' => '<string>',
'ScriptLocation' => '<string>',
],
'Connections' => [
'Connections' => ['<string>', ...],
],
'CreatedOn' => <DateTime>,
'DefaultArguments' => ['<string>', ...],
'Description' => '<string>',
'ExecutionProperty' => [
'MaxConcurrentRuns' => <integer>,
],
'LastModifiedOn' => <DateTime>,
'LogUri' => '<string>',
'MaxCapacity' => <float>,
'MaxRetries' => <integer>,
'Name' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'Role' => '<string>',
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
'NextToken' => '<string>',
]
Result Details
Members
- Jobs
-
- Type: Array of Job structures
A list of job definitions.
- NextToken
-
- Type: string
A continuation token, if not all job definitions have yet been returned.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetMapping
$result = $client->getMapping([/* ... */]); $promise = $client->getMappingAsync([/* ... */]);
Creates mappings.
Parameter Syntax
$result = $client->getMapping([
'Location' => [
'DynamoDB' => [
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
'Jdbc' => [
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
'S3' => [
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
],
'Sinks' => [
[
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
],
// ...
],
'Source' => [ // REQUIRED
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
],
]);
Parameter Details
Members
- Location
-
- Type: Location structure
Parameters for the mapping.
- Sinks
-
- Type: Array of CatalogEntry structures
A list of target tables.
- Source
-
- Required: Yes
- Type: CatalogEntry structure
Specifies the source table.
Result Syntax
[
'Mapping' => [
[
'SourcePath' => '<string>',
'SourceTable' => '<string>',
'SourceType' => '<string>',
'TargetPath' => '<string>',
'TargetTable' => '<string>',
'TargetType' => '<string>',
],
// ...
],
]
Result Details
Members
- Mapping
-
- Required: Yes
- Type: Array of MappingEntry structures
A list of mappings to the specified targets.
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
A specified entity does not exist
GetPartition
$result = $client->getPartition([/* ... */]); $promise = $client->getPartitionAsync([/* ... */]);
Retrieves information about a specified partition.
Parameter Syntax
$result = $client->getPartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionValues' => ['<string>', ...], // REQUIRED
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the partition in question resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the partition resides.
- PartitionValues
-
- Required: Yes
- Type: Array of strings
The values that define the partition.
- TableName
-
- Required: Yes
- Type: string
The name of the partition's table.
Result Syntax
[
'Partition' => [
'CreationTime' => <DateTime>,
'DatabaseName' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Parameters' => ['<string>', ...],
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableName' => '<string>',
'Values' => ['<string>', ...],
],
]
Result Details
Members
- Partition
-
- Type: Partition structure
The requested information, in the form of a
Partitionobject.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetPartitions
$result = $client->getPartitions([/* ... */]); $promise = $client->getPartitionsAsync([/* ... */]);
Retrieves information about the partitions in a table.
Parameter Syntax
$result = $client->getPartitions([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'Expression' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
'Segment' => [
'SegmentNumber' => <integer>, // REQUIRED
'TotalSegments' => <integer>, // REQUIRED
],
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the partitions reside.
- Expression
-
- Type: string
An expression filtering the partitions to be returned.
The expression uses SQL syntax similar to the SQL
WHEREfilter clause. The SQL statement parser JSQLParser parses the expression.Operators: The following are the operators that you can use in the
ExpressionAPI call:- =
-
Checks if the values of the two operands are equal or not; if yes, then the condition becomes true.
Example: Assume 'variable a' holds 10 and 'variable b' holds 20.
(a = b) is not true.
- < >
-
Checks if the values of two operands are equal or not; if the values are not equal, then the condition becomes true.
Example: (a < > b) is true.
- >
-
Checks if the value of the left operand is greater than the value of the right operand; if yes, then the condition becomes true.
Example: (a > b) is not true.
- <
-
Checks if the value of the left operand is less than the value of the right operand; if yes, then the condition becomes true.
Example: (a < b) is true.
- >=
-
Checks if the value of the left operand is greater than or equal to the value of the right operand; if yes, then the condition becomes true.
Example: (a >= b) is not true.
- <=
-
Checks if the value of the left operand is less than or equal to the value of the right operand; if yes, then the condition becomes true.
Example: (a <= b) is true.
- AND, OR, IN, BETWEEN, LIKE, NOT, IS NULL
-
Logical operators.
Supported Partition Key Types: The following are the the supported partition keys.
-
string -
date -
timestamp -
int -
bigint -
long -
tinyint -
smallint -
decimal
If an invalid type is encountered, an exception is thrown.
The following list shows the valid operators on each type. When you define a crawler, the
partitionKeytype is created as aSTRING, to be compatible with the catalog partitions.Sample API Call:
- MaxResults
-
- Type: int
The maximum number of partitions to return in a single response.
- NextToken
-
- Type: string
A continuation token, if this is not the first call to retrieve these partitions.
- Segment
-
- Type: Segment structure
The segment of the table's partitions to scan in this request.
- TableName
-
- Required: Yes
- Type: string
The name of the partitions' table.
Result Syntax
[
'NextToken' => '<string>',
'Partitions' => [
[
'CreationTime' => <DateTime>,
'DatabaseName' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Parameters' => ['<string>', ...],
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableName' => '<string>',
'Values' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, if the returned list of partitions does not does not include the last one.
- Partitions
-
- Type: Array of Partition structures
A list of requested partitions.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
The operation timed out.
-
An internal service error occurred.
-
An encryption operation failed.
GetPlan
$result = $client->getPlan([/* ... */]); $promise = $client->getPlanAsync([/* ... */]);
Gets code to perform a specified mapping.
Parameter Syntax
$result = $client->getPlan([
'Language' => 'PYTHON|SCALA',
'Location' => [
'DynamoDB' => [
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
'Jdbc' => [
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
'S3' => [
[
'Name' => '<string>', // REQUIRED
'Param' => true || false,
'Value' => '<string>', // REQUIRED
],
// ...
],
],
'Mapping' => [ // REQUIRED
[
'SourcePath' => '<string>',
'SourceTable' => '<string>',
'SourceType' => '<string>',
'TargetPath' => '<string>',
'TargetTable' => '<string>',
'TargetType' => '<string>',
],
// ...
],
'Sinks' => [
[
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
],
// ...
],
'Source' => [ // REQUIRED
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
],
]);
Parameter Details
Members
- Language
-
- Type: string
The programming language of the code to perform the mapping.
- Location
-
- Type: Location structure
The parameters for the mapping.
- Mapping
-
- Required: Yes
- Type: Array of MappingEntry structures
The list of mappings from a source table to target tables.
- Sinks
-
- Type: Array of CatalogEntry structures
The target tables.
- Source
-
- Required: Yes
- Type: CatalogEntry structure
The source table.
Result Syntax
[
'PythonScript' => '<string>',
'ScalaCode' => '<string>',
]
Result Details
Members
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
GetResourcePolicy
$result = $client->getResourcePolicy([/* ... */]); $promise = $client->getResourcePolicyAsync([/* ... */]);
Retrieves a specified resource policy.
Parameter Syntax
$result = $client->getResourcePolicy([ ]);
Parameter Details
Members
Result Syntax
[
'CreateTime' => <DateTime>,
'PolicyHash' => '<string>',
'PolicyInJson' => '<string>',
'UpdateTime' => <DateTime>,
]
Result Details
Members
- CreateTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time at which the policy was created.
- PolicyHash
-
- Type: string
Contains the hash value associated with this policy.
- PolicyInJson
-
- Type: string
Contains the requested policy document, in JSON format.
- UpdateTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time at which the policy was last updated.
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
GetSecurityConfiguration
$result = $client->getSecurityConfiguration([/* ... */]); $promise = $client->getSecurityConfigurationAsync([/* ... */]);
Retrieves a specified security configuration.
Parameter Syntax
$result = $client->getSecurityConfiguration([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'SecurityConfiguration' => [
'CreatedTimeStamp' => <DateTime>,
'EncryptionConfiguration' => [
'CloudWatchEncryption' => [
'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
'KmsKeyArn' => '<string>',
],
'JobBookmarksEncryption' => [
'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
'KmsKeyArn' => '<string>',
],
'S3Encryption' => [
[
'KmsKeyArn' => '<string>',
'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
],
// ...
],
],
'Name' => '<string>',
],
]
Result Details
Members
- SecurityConfiguration
-
- Type: SecurityConfiguration structure
The requested security configuration
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
GetSecurityConfigurations
$result = $client->getSecurityConfigurations([/* ... */]); $promise = $client->getSecurityConfigurationsAsync([/* ... */]);
Retrieves a list of all security configurations.
Parameter Syntax
$result = $client->getSecurityConfigurations([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'NextToken' => '<string>',
'SecurityConfigurations' => [
[
'CreatedTimeStamp' => <DateTime>,
'EncryptionConfiguration' => [
'CloudWatchEncryption' => [
'CloudWatchEncryptionMode' => 'DISABLED|SSE-KMS',
'KmsKeyArn' => '<string>',
],
'JobBookmarksEncryption' => [
'JobBookmarksEncryptionMode' => 'DISABLED|CSE-KMS',
'KmsKeyArn' => '<string>',
],
'S3Encryption' => [
[
'KmsKeyArn' => '<string>',
'S3EncryptionMode' => 'DISABLED|SSE-KMS|SSE-S3',
],
// ...
],
],
'Name' => '<string>',
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, if there are more security configurations to return.
- SecurityConfigurations
-
- Type: Array of SecurityConfiguration structures
A list of security configurations.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
GetTable
$result = $client->getTable([/* ... */]); $promise = $client->getTableAsync([/* ... */]);
Retrieves the Table definition in a Data Catalog for a specified table.
Parameter Syntax
$result = $client->getTable([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.
- Name
-
- Required: Yes
- Type: string
The name of the table for which to retrieve the definition. For Hive compatibility, this name is entirely lowercase.
Result Syntax
[
'Table' => [
'CreateTime' => <DateTime>,
'CreatedBy' => '<string>',
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Name' => '<string>',
'Owner' => '<string>',
'Parameters' => ['<string>', ...],
'PartitionKeys' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Retention' => <integer>,
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableType' => '<string>',
'UpdateTime' => <DateTime>,
'ViewExpandedText' => '<string>',
'ViewOriginalText' => '<string>',
],
]
Result Details
Members
- Table
-
- Type: Table structure
The
Tableobject that defines the specified table.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetTableVersion
$result = $client->getTableVersion([/* ... */]); $promise = $client->getTableVersionAsync([/* ... */]);
Retrieves a specified version of a table.
Parameter Syntax
$result = $client->getTableVersion([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'TableName' => '<string>', // REQUIRED
'VersionId' => '<string>',
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.
- TableName
-
- Required: Yes
- Type: string
The name of the table. For Hive compatibility, this name is entirely lowercase.
- VersionId
-
- Type: string
The ID value of the table version to be retrieved. A
VersionIDis a string representation of an integer. Each version is incremented by 1.
Result Syntax
[
'TableVersion' => [
'Table' => [
'CreateTime' => <DateTime>,
'CreatedBy' => '<string>',
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Name' => '<string>',
'Owner' => '<string>',
'Parameters' => ['<string>', ...],
'PartitionKeys' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Retention' => <integer>,
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableType' => '<string>',
'UpdateTime' => <DateTime>,
'ViewExpandedText' => '<string>',
'ViewOriginalText' => '<string>',
],
'VersionId' => '<string>',
],
]
Result Details
Members
- TableVersion
-
- Type: TableVersion structure
The requested table version.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetTableVersions
$result = $client->getTableVersions([/* ... */]); $promise = $client->getTableVersionsAsync([/* ... */]);
Retrieves a list of strings that identify available versions of a specified table.
Parameter Syntax
$result = $client->getTableVersions([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'MaxResults' => <integer>,
'NextToken' => '<string>',
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.
- MaxResults
-
- Type: int
The maximum number of table versions to return in one response.
- NextToken
-
- Type: string
A continuation token, if this is not the first call.
- TableName
-
- Required: Yes
- Type: string
The name of the table. For Hive compatibility, this name is entirely lowercase.
Result Syntax
[
'NextToken' => '<string>',
'TableVersions' => [
[
'Table' => [
'CreateTime' => <DateTime>,
'CreatedBy' => '<string>',
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Name' => '<string>',
'Owner' => '<string>',
'Parameters' => ['<string>', ...],
'PartitionKeys' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Retention' => <integer>,
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableType' => '<string>',
'UpdateTime' => <DateTime>,
'ViewExpandedText' => '<string>',
'ViewOriginalText' => '<string>',
],
'VersionId' => '<string>',
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, if the list of available versions does not include the last one.
- TableVersions
-
- Type: Array of TableVersion structures
A list of strings identifying available versions of the specified table.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetTables
$result = $client->getTables([/* ... */]); $promise = $client->getTablesAsync([/* ... */]);
Retrieves the definitions of some or all of the tables in a given Database.
Parameter Syntax
$result = $client->getTables([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'Expression' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The database in the catalog whose tables to list. For Hive compatibility, this name is entirely lowercase.
- Expression
-
- Type: string
A regular expression pattern. If present, only those tables whose names match the pattern are returned.
- MaxResults
-
- Type: int
The maximum number of tables to return in a single response.
- NextToken
-
- Type: string
A continuation token, included if this is a continuation call.
Result Syntax
[
'NextToken' => '<string>',
'TableList' => [
[
'CreateTime' => <DateTime>,
'CreatedBy' => '<string>',
'DatabaseName' => '<string>',
'Description' => '<string>',
'LastAccessTime' => <DateTime>,
'LastAnalyzedTime' => <DateTime>,
'Name' => '<string>',
'Owner' => '<string>',
'Parameters' => ['<string>', ...],
'PartitionKeys' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Retention' => <integer>,
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>',
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>',
'SortOrder' => <integer>,
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableType' => '<string>',
'UpdateTime' => <DateTime>,
'ViewExpandedText' => '<string>',
'ViewOriginalText' => '<string>',
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, present if the current list segment is not the last.
- TableList
-
- Type: Array of Table structures
A list of the requested
Tableobjects.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
The operation timed out.
-
An internal service error occurred.
-
An encryption operation failed.
GetTags
$result = $client->getTags([/* ... */]); $promise = $client->getTagsAsync([/* ... */]);
Retrieves a list of tags associated with a resource.
Parameter Syntax
$result = $client->getTags([
'ResourceArn' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Tags' => ['<string>', ...],
]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
A specified entity does not exist
GetTrigger
$result = $client->getTrigger([/* ... */]); $promise = $client->getTriggerAsync([/* ... */]);
Retrieves the definition of a trigger.
Parameter Syntax
$result = $client->getTrigger([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
]
Result Details
Members
- Trigger
-
- Type: Trigger structure
The requested trigger definition.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
GetTriggers
$result = $client->getTriggers([/* ... */]); $promise = $client->getTriggersAsync([/* ... */]);
Gets all the triggers associated with a job.
Parameter Syntax
$result = $client->getTriggers([
'DependentJobName' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
- DependentJobName
-
- Type: string
The name of the job to retrieve triggers for. The trigger that can start this job is returned, and if there is no such trigger, all triggers are returned.
- MaxResults
-
- Type: int
The maximum size of the response.
- NextToken
-
- Type: string
A continuation token, if this is a continuation call.
Result Syntax
[
'NextToken' => '<string>',
'Triggers' => [
[
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, if not all the requested triggers have yet been returned.
- Triggers
-
- Type: Array of Trigger structures
A list of triggers for the specified job.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
GetUserDefinedFunction
$result = $client->getUserDefinedFunction([/* ... */]); $promise = $client->getUserDefinedFunctionAsync([/* ... */]);
Retrieves a specified function definition from the Data Catalog.
Parameter Syntax
$result = $client->getUserDefinedFunction([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'FunctionName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the function to be retrieved is located. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the function is located.
- FunctionName
-
- Required: Yes
- Type: string
The name of the function.
Result Syntax
[
'UserDefinedFunction' => [
'ClassName' => '<string>',
'CreateTime' => <DateTime>,
'FunctionName' => '<string>',
'OwnerName' => '<string>',
'OwnerType' => 'USER|ROLE|GROUP',
'ResourceUris' => [
[
'ResourceType' => 'JAR|FILE|ARCHIVE',
'Uri' => '<string>',
],
// ...
],
],
]
Result Details
Members
- UserDefinedFunction
-
- Type: UserDefinedFunction structure
The requested function definition.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
GetUserDefinedFunctions
$result = $client->getUserDefinedFunctions([/* ... */]); $promise = $client->getUserDefinedFunctionsAsync([/* ... */]);
Retrieves a multiple function definitions from the Data Catalog.
Parameter Syntax
$result = $client->getUserDefinedFunctions([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'MaxResults' => <integer>,
'NextToken' => '<string>',
'Pattern' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the functions to be retrieved are located. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the functions are located.
- MaxResults
-
- Type: int
The maximum number of functions to return in one response.
- NextToken
-
- Type: string
A continuation token, if this is a continuation call.
- Pattern
-
- Required: Yes
- Type: string
An optional function-name pattern string that filters the function definitions returned.
Result Syntax
[
'NextToken' => '<string>',
'UserDefinedFunctions' => [
[
'ClassName' => '<string>',
'CreateTime' => <DateTime>,
'FunctionName' => '<string>',
'OwnerName' => '<string>',
'OwnerType' => 'USER|ROLE|GROUP',
'ResourceUris' => [
[
'ResourceType' => 'JAR|FILE|ARCHIVE',
'Uri' => '<string>',
],
// ...
],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, if the list of functions returned does not include the last requested function.
- UserDefinedFunctions
-
- Type: Array of UserDefinedFunction structures
A list of requested function definitions.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
The operation timed out.
-
An internal service error occurred.
-
An encryption operation failed.
GetWorkflow
$result = $client->getWorkflow([/* ... */]); $promise = $client->getWorkflowAsync([/* ... */]);
Retrieves resource metadata for a workflow.
Parameter Syntax
$result = $client->getWorkflow([
'IncludeGraph' => true || false,
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Workflow' => [
'CreatedOn' => <DateTime>,
'DefaultRunProperties' => ['<string>', ...],
'Description' => '<string>',
'Graph' => [
'Edges' => [
[
'DestinationId' => '<string>',
'SourceId' => '<string>',
],
// ...
],
'Nodes' => [
[
'CrawlerDetails' => [
'Crawls' => [
[
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
],
// ...
],
],
'JobDetails' => [
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
],
'Name' => '<string>',
'TriggerDetails' => [
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
],
'Type' => 'CRAWLER|JOB|TRIGGER',
'UniqueId' => '<string>',
],
// ...
],
],
'LastModifiedOn' => <DateTime>,
'LastRun' => [
'CompletedOn' => <DateTime>,
'Graph' => [
'Edges' => [
[
'DestinationId' => '<string>',
'SourceId' => '<string>',
],
// ...
],
'Nodes' => [
[
'CrawlerDetails' => [
'Crawls' => [
[
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
],
// ...
],
],
'JobDetails' => [
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
],
'Name' => '<string>',
'TriggerDetails' => [
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
],
'Type' => 'CRAWLER|JOB|TRIGGER',
'UniqueId' => '<string>',
],
// ...
],
],
'Name' => '<string>',
'StartedOn' => <DateTime>,
'Statistics' => [
'FailedActions' => <integer>,
'RunningActions' => <integer>,
'StoppedActions' => <integer>,
'SucceededActions' => <integer>,
'TimeoutActions' => <integer>,
'TotalActions' => <integer>,
],
'Status' => 'RUNNING|COMPLETED',
'WorkflowRunId' => '<string>',
'WorkflowRunProperties' => ['<string>', ...],
],
'Name' => '<string>',
],
]
Result Details
Members
- Workflow
-
- Type: Workflow structure
The resource metadata for the workflow.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetWorkflowRun
$result = $client->getWorkflowRun([/* ... */]); $promise = $client->getWorkflowRunAsync([/* ... */]);
Retrieves the metadata for a given workflow run.
Parameter Syntax
$result = $client->getWorkflowRun([
'IncludeGraph' => true || false,
'Name' => '<string>', // REQUIRED
'RunId' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Run' => [
'CompletedOn' => <DateTime>,
'Graph' => [
'Edges' => [
[
'DestinationId' => '<string>',
'SourceId' => '<string>',
],
// ...
],
'Nodes' => [
[
'CrawlerDetails' => [
'Crawls' => [
[
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
],
// ...
],
],
'JobDetails' => [
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
],
'Name' => '<string>',
'TriggerDetails' => [
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
],
'Type' => 'CRAWLER|JOB|TRIGGER',
'UniqueId' => '<string>',
],
// ...
],
],
'Name' => '<string>',
'StartedOn' => <DateTime>,
'Statistics' => [
'FailedActions' => <integer>,
'RunningActions' => <integer>,
'StoppedActions' => <integer>,
'SucceededActions' => <integer>,
'TimeoutActions' => <integer>,
'TotalActions' => <integer>,
],
'Status' => 'RUNNING|COMPLETED',
'WorkflowRunId' => '<string>',
'WorkflowRunProperties' => ['<string>', ...],
],
]
Result Details
Members
- Run
-
- Type: WorkflowRun structure
The requested workflow run metadata.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetWorkflowRunProperties
$result = $client->getWorkflowRunProperties([/* ... */]); $promise = $client->getWorkflowRunPropertiesAsync([/* ... */]);
Retrieves the workflow run properties which were set during the run.
Parameter Syntax
$result = $client->getWorkflowRunProperties([
'Name' => '<string>', // REQUIRED
'RunId' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'RunProperties' => ['<string>', ...],
]
Result Details
Members
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
GetWorkflowRuns
$result = $client->getWorkflowRuns([/* ... */]); $promise = $client->getWorkflowRunsAsync([/* ... */]);
Retrieves metadata for all runs of a given workflow.
Parameter Syntax
$result = $client->getWorkflowRuns([
'IncludeGraph' => true || false,
'MaxResults' => <integer>,
'Name' => '<string>', // REQUIRED
'NextToken' => '<string>',
]);
Parameter Details
Members
- IncludeGraph
-
- Type: boolean
Specifies whether to include the workflow graph in response or not.
- MaxResults
-
- Type: int
The maximum number of workflow runs to be included in the response.
- Name
-
- Required: Yes
- Type: string
Name of the workflow whose metadata of runs should be returned.
- NextToken
-
- Type: string
The maximum size of the response.
Result Syntax
[
'NextToken' => '<string>',
'Runs' => [
[
'CompletedOn' => <DateTime>,
'Graph' => [
'Edges' => [
[
'DestinationId' => '<string>',
'SourceId' => '<string>',
],
// ...
],
'Nodes' => [
[
'CrawlerDetails' => [
'Crawls' => [
[
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'LogGroup' => '<string>',
'LogStream' => '<string>',
'StartedOn' => <DateTime>,
'State' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
],
// ...
],
],
'JobDetails' => [
'JobRuns' => [
[
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'Attempt' => <integer>,
'CompletedOn' => <DateTime>,
'ErrorMessage' => '<string>',
'ExecutionTime' => <integer>,
'Id' => '<string>',
'JobName' => '<string>',
'JobRunState' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
'LastModifiedOn' => <DateTime>,
'LogGroupName' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'PredecessorRuns' => [
[
'JobName' => '<string>',
'RunId' => '<string>',
],
// ...
],
'PreviousRunId' => '<string>',
'SecurityConfiguration' => '<string>',
'StartedOn' => <DateTime>,
'Timeout' => <integer>,
'TriggerName' => '<string>',
'WorkerType' => 'Standard|G.1X|G.2X',
],
// ...
],
],
'Name' => '<string>',
'TriggerDetails' => [
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
],
'Type' => 'CRAWLER|JOB|TRIGGER',
'UniqueId' => '<string>',
],
// ...
],
],
'Name' => '<string>',
'StartedOn' => <DateTime>,
'Statistics' => [
'FailedActions' => <integer>,
'RunningActions' => <integer>,
'StoppedActions' => <integer>,
'SucceededActions' => <integer>,
'TimeoutActions' => <integer>,
'TotalActions' => <integer>,
],
'Status' => 'RUNNING|COMPLETED',
'WorkflowRunId' => '<string>',
'WorkflowRunProperties' => ['<string>', ...],
],
// ...
],
]
Result Details
Members
- NextToken
-
- Type: string
A continuation token, if not all requested workflow runs have been returned.
- Runs
-
- Type: Array of WorkflowRun structures
A list of workflow run metadata objects.
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
ImportCatalogToGlue
$result = $client->importCatalogToGlue([/* ... */]); $promise = $client->importCatalogToGlueAsync([/* ... */]);
Imports an existing Athena Data Catalog to AWS Glue
Parameter Syntax
$result = $client->importCatalogToGlue([
'CatalogId' => '<string>',
]);
Parameter Details
Members
Result Syntax
[]
Result Details
Errors
-
An internal service error occurred.
-
The operation timed out.
ListCrawlers
$result = $client->listCrawlers([/* ... */]); $promise = $client->listCrawlersAsync([/* ... */]);
Retrieves the names of all crawler resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.
This operation takes the optional Tags field, which you can use as a filter on the response so that tagged resources can be retrieved as a group. If you choose to use tags filtering, only resources with the tag are retrieved.
Parameter Syntax
$result = $client->listCrawlers([
'MaxResults' => <integer>,
'NextToken' => '<string>',
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
Result Syntax
[
'CrawlerNames' => ['<string>', ...],
'NextToken' => '<string>',
]
Result Details
Members
Errors
-
The operation timed out.
ListDevEndpoints
$result = $client->listDevEndpoints([/* ... */]); $promise = $client->listDevEndpointsAsync([/* ... */]);
Retrieves the names of all DevEndpoint resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.
This operation takes the optional Tags field, which you can use as a filter on the response so that tagged resources can be retrieved as a group. If you choose to use tags filtering, only resources with the tag are retrieved.
Parameter Syntax
$result = $client->listDevEndpoints([
'MaxResults' => <integer>,
'NextToken' => '<string>',
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
Result Syntax
[
'DevEndpointNames' => ['<string>', ...],
'NextToken' => '<string>',
]
Result Details
Members
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
ListJobs
$result = $client->listJobs([/* ... */]); $promise = $client->listJobsAsync([/* ... */]);
Retrieves the names of all job resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.
This operation takes the optional Tags field, which you can use as a filter on the response so that tagged resources can be retrieved as a group. If you choose to use tags filtering, only resources with the tag are retrieved.
Parameter Syntax
$result = $client->listJobs([
'MaxResults' => <integer>,
'NextToken' => '<string>',
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
Result Syntax
[
'JobNames' => ['<string>', ...],
'NextToken' => '<string>',
]
Result Details
Members
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
ListTriggers
$result = $client->listTriggers([/* ... */]); $promise = $client->listTriggersAsync([/* ... */]);
Retrieves the names of all trigger resources in this AWS account, or the resources with the specified tag. This operation allows you to see which resources are available in your account, and their names.
This operation takes the optional Tags field, which you can use as a filter on the response so that tagged resources can be retrieved as a group. If you choose to use tags filtering, only resources with the tag are retrieved.
Parameter Syntax
$result = $client->listTriggers([
'DependentJobName' => '<string>',
'MaxResults' => <integer>,
'NextToken' => '<string>',
'Tags' => ['<string>', ...],
]);
Parameter Details
Members
- DependentJobName
-
- Type: string
The name of the job for which to retrieve triggers. The trigger that can start this job is returned. If there is no such trigger, all triggers are returned.
- MaxResults
-
- Type: int
The maximum size of a list to return.
- NextToken
-
- Type: string
A continuation token, if this is a continuation request.
- Tags
-
- Type: Associative array of custom strings keys (TagKey) to strings
Specifies to return only these tagged resources.
Result Syntax
[
'NextToken' => '<string>',
'TriggerNames' => ['<string>', ...],
]
Result Details
Members
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
ListWorkflows
$result = $client->listWorkflows([/* ... */]); $promise = $client->listWorkflowsAsync([/* ... */]);
Lists names of workflows created in the account.
Parameter Syntax
$result = $client->listWorkflows([
'MaxResults' => <integer>,
'NextToken' => '<string>',
]);
Parameter Details
Members
Result Syntax
[
'NextToken' => '<string>',
'Workflows' => ['<string>', ...],
]
Result Details
Members
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
PutDataCatalogEncryptionSettings
$result = $client->putDataCatalogEncryptionSettings([/* ... */]); $promise = $client->putDataCatalogEncryptionSettingsAsync([/* ... */]);
Sets the security configuration for a specified catalog. After the configuration has been set, the specified encryption is applied to every catalog write thereafter.
Parameter Syntax
$result = $client->putDataCatalogEncryptionSettings([
'CatalogId' => '<string>',
'DataCatalogEncryptionSettings' => [ // REQUIRED
'ConnectionPasswordEncryption' => [
'AwsKmsKeyId' => '<string>',
'ReturnConnectionPasswordEncrypted' => true || false, // REQUIRED
],
'EncryptionAtRest' => [
'CatalogEncryptionMode' => 'DISABLED|SSE-KMS', // REQUIRED
'SseAwsKmsKeyId' => '<string>',
],
],
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog for which to set the security configuration. If none is provided, the AWS account ID is used by default.
- DataCatalogEncryptionSettings
-
- Required: Yes
- Type: DataCatalogEncryptionSettings structure
The security configuration to set.
Result Syntax
[]
Result Details
Errors
-
An internal service error occurred.
-
The input provided was not valid.
-
The operation timed out.
PutResourcePolicy
$result = $client->putResourcePolicy([/* ... */]); $promise = $client->putResourcePolicyAsync([/* ... */]);
Sets the Data Catalog resource policy for access control.
Parameter Syntax
$result = $client->putResourcePolicy([
'PolicyExistsCondition' => 'MUST_EXIST|NOT_EXIST|NONE',
'PolicyHashCondition' => '<string>',
'PolicyInJson' => '<string>', // REQUIRED
]);
Parameter Details
Members
- PolicyExistsCondition
-
- Type: string
A value of
MUST_EXISTis used to update a policy. A value ofNOT_EXISTis used to create a new policy. If a value ofNONEor a null value is used, the call will not depend on the existence of a policy. - PolicyHashCondition
-
- Type: string
The hash value returned when the previous policy was set using
PutResourcePolicy. Its purpose is to prevent concurrent modifications of a policy. Do not use this parameter if no previous policy has been set. - PolicyInJson
-
- Required: Yes
- Type: string
Contains the policy document to set, in JSON format.
Result Syntax
[
'PolicyHash' => '<string>',
]
Result Details
Members
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
-
ConditionCheckFailureException:
A specified condition was not satisfied.
PutWorkflowRunProperties
$result = $client->putWorkflowRunProperties([/* ... */]); $promise = $client->putWorkflowRunPropertiesAsync([/* ... */]);
Puts the specified workflow run properties for the given workflow run. If a property already exists for the specified run, then it overrides the value otherwise adds the property to existing properties.
Parameter Syntax
$result = $client->putWorkflowRunProperties([
'Name' => '<string>', // REQUIRED
'RunId' => '<string>', // REQUIRED
'RunProperties' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
- Name
-
- Required: Yes
- Type: string
Name of the workflow which was run.
- RunId
-
- Required: Yes
- Type: string
The ID of the workflow run for which the run properties should be updated.
- RunProperties
-
- Required: Yes
- Type: Associative array of custom strings keys (IdString) to strings
The properties to put for the specified run.
Result Syntax
[]
Result Details
Errors
-
A resource to be created or added already exists.
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
ResetJobBookmark
$result = $client->resetJobBookmark([/* ... */]); $promise = $client->resetJobBookmarkAsync([/* ... */]);
Resets a bookmark entry.
Parameter Syntax
$result = $client->resetJobBookmark([
'JobName' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'JobBookmarkEntry' => [
'Attempt' => <integer>,
'JobBookmark' => '<string>',
'JobName' => '<string>',
'Run' => <integer>,
'Version' => <integer>,
],
]
Result Details
Members
- JobBookmarkEntry
-
- Type: JobBookmarkEntry structure
The reset bookmark entry.
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
StartCrawler
$result = $client->startCrawler([/* ... */]); $promise = $client->startCrawlerAsync([/* ... */]);
Starts a crawl using the specified crawler, regardless of what is scheduled. If the crawler is already running, returns a CrawlerRunningException.
Parameter Syntax
$result = $client->startCrawler([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The operation cannot be performed because the crawler is already running.
-
The operation timed out.
StartCrawlerSchedule
$result = $client->startCrawlerSchedule([/* ... */]); $promise = $client->startCrawlerScheduleAsync([/* ... */]);
Changes the schedule state of the specified crawler to SCHEDULED, unless the crawler is already running or the schedule state is already SCHEDULED.
Parameter Syntax
$result = $client->startCrawlerSchedule([
'CrawlerName' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The specified scheduler is already running.
-
SchedulerTransitioningException:
The specified scheduler is transitioning.
-
There is no applicable schedule.
-
The operation timed out.
StartJobRun
$result = $client->startJobRun([/* ... */]); $promise = $client->startJobRunAsync([/* ... */]);
Starts a job run using a job definition.
Parameter Syntax
$result = $client->startJobRun([
'AllocatedCapacity' => <integer>,
'Arguments' => ['<string>', ...],
'JobName' => '<string>', // REQUIRED
'JobRunId' => '<string>',
'MaxCapacity' => <float>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
'WorkerType' => '<string>',
]);
Parameter Details
Members
- AllocatedCapacity
-
- Type: int
This field is deprecated. Use
MaxCapacityinstead.The number of AWS Glue data processing units (DPUs) to allocate to this JobRun. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
- Arguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The job arguments specifically for this run. For this job run, they replace the default arguments set in the job definition itself.
You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.
For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.
- JobName
-
- Required: Yes
- Type: string
The name of the job definition to use.
- JobRunId
-
- Type: string
The ID of a previous
JobRunto retry. - MaxCapacity
-
- Type: double
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
Do not set
Max Capacityif usingWorkerTypeandNumberOfWorkers.The value that can be allocated for
MaxCapacitydepends on whether you are running a Python shell job, or an Apache Spark ETL job:-
When you specify a Python shell job (
JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU. -
When you specify an Apache Spark ETL job (
JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.
- NotificationProperty
-
- Type: NotificationProperty structure
Specifies configuration properties of a job run notification.
- NumberOfWorkers
-
- Type: int
The number of workers of a defined
workerTypethat are allocated when a job runs.The maximum number of workers you can define are 299 for
G.1X, and 149 forG.2X. - SecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used with this job run. - Timeout
-
- Type: int
The
JobRuntimeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and entersTIMEOUTstatus. The default is 2,880 minutes (48 hours). This overrides the timeout value set in the parent job. - WorkerType
-
- Type: string
The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.
-
For the
Standardworker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker. -
For the
G.1Xworker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker. -
For the
G.2Xworker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.
Result Syntax
[
'JobRunId' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentRunsExceededException:
Too many jobs are being run concurrently.
StartTrigger
$result = $client->startTrigger([/* ... */]); $promise = $client->startTriggerAsync([/* ... */]);
Starts an existing trigger. See Triggering Jobs for information about how different types of trigger are started.
Parameter Syntax
$result = $client->startTrigger([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
A specified entity does not exist
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentRunsExceededException:
Too many jobs are being run concurrently.
StartWorkflowRun
$result = $client->startWorkflowRun([/* ... */]); $promise = $client->startWorkflowRunAsync([/* ... */]);
Starts a new run of the specified workflow.
Parameter Syntax
$result = $client->startWorkflowRun([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'RunId' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
ConcurrentRunsExceededException:
Too many jobs are being run concurrently.
StopCrawler
$result = $client->stopCrawler([/* ... */]); $promise = $client->stopCrawlerAsync([/* ... */]);
If the specified crawler is running, stops the crawl.
Parameter Syntax
$result = $client->stopCrawler([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The specified crawler is not running.
-
The specified crawler is stopping.
-
The operation timed out.
StopCrawlerSchedule
$result = $client->stopCrawlerSchedule([/* ... */]); $promise = $client->stopCrawlerScheduleAsync([/* ... */]);
Sets the schedule state of the specified crawler to NOT_SCHEDULED, but does not stop the crawler if it is already running.
Parameter Syntax
$result = $client->stopCrawlerSchedule([
'CrawlerName' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The specified scheduler is not running.
-
SchedulerTransitioningException:
The specified scheduler is transitioning.
-
The operation timed out.
StopTrigger
$result = $client->stopTrigger([/* ... */]); $promise = $client->stopTriggerAsync([/* ... */]);
Stops a specified trigger.
Parameter Syntax
$result = $client->stopTrigger([
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
A specified entity does not exist
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
TagResource
$result = $client->tagResource([/* ... */]); $promise = $client->tagResourceAsync([/* ... */]);
Adds tags to a resource. A tag is a label you can assign to an AWS resource. In AWS Glue, you can tag only certain resources. For information about what resources you can tag, see AWS Tags in AWS Glue.
Parameter Syntax
$result = $client->tagResource([
'ResourceArn' => '<string>', // REQUIRED
'TagsToAdd' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
- ResourceArn
-
- Required: Yes
- Type: string
The ARN of the AWS Glue resource to which to add the tags. For more information about AWS Glue resource ARNs, see the AWS Glue ARN string pattern.
- TagsToAdd
-
- Required: Yes
- Type: Associative array of custom strings keys (TagKey) to strings
Tags to add to this resource.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
A specified entity does not exist
UntagResource
$result = $client->untagResource([/* ... */]); $promise = $client->untagResourceAsync([/* ... */]);
Removes tags from a resource.
Parameter Syntax
$result = $client->untagResource([
'ResourceArn' => '<string>', // REQUIRED
'TagsToRemove' => ['<string>', ...], // REQUIRED
]);
Parameter Details
Members
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
A specified entity does not exist
UpdateClassifier
$result = $client->updateClassifier([/* ... */]); $promise = $client->updateClassifierAsync([/* ... */]);
Modifies an existing classifier (a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field is present).
Parameter Syntax
$result = $client->updateClassifier([
'CsvClassifier' => [
'AllowSingleColumn' => true || false,
'ContainsHeader' => 'UNKNOWN|PRESENT|ABSENT',
'Delimiter' => '<string>',
'DisableValueTrimming' => true || false,
'Header' => ['<string>', ...],
'Name' => '<string>', // REQUIRED
'QuoteSymbol' => '<string>',
],
'GrokClassifier' => [
'Classification' => '<string>',
'CustomPatterns' => '<string>',
'GrokPattern' => '<string>',
'Name' => '<string>', // REQUIRED
],
'JsonClassifier' => [
'JsonPath' => '<string>',
'Name' => '<string>', // REQUIRED
],
'XMLClassifier' => [
'Classification' => '<string>',
'Name' => '<string>', // REQUIRED
'RowTag' => '<string>',
],
]);
Parameter Details
Members
- CsvClassifier
-
- Type: UpdateCsvClassifierRequest structure
A
CsvClassifierobject with updated fields. - GrokClassifier
-
- Type: UpdateGrokClassifierRequest structure
A
GrokClassifierobject with updated fields. - JsonClassifier
-
- Type: UpdateJsonClassifierRequest structure
A
JsonClassifierobject with updated fields. - XMLClassifier
-
- Type: UpdateXMLClassifierRequest structure
An
XMLClassifierobject with updated fields.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
There was a version conflict.
-
A specified entity does not exist
-
The operation timed out.
UpdateConnection
$result = $client->updateConnection([/* ... */]); $promise = $client->updateConnectionAsync([/* ... */]);
Updates a connection definition in the Data Catalog.
Parameter Syntax
$result = $client->updateConnection([
'CatalogId' => '<string>',
'ConnectionInput' => [ // REQUIRED
'ConnectionProperties' => ['<string>', ...], // REQUIRED
'ConnectionType' => 'JDBC|SFTP', // REQUIRED
'Description' => '<string>',
'MatchCriteria' => ['<string>', ...],
'Name' => '<string>', // REQUIRED
'PhysicalConnectionRequirements' => [
'AvailabilityZone' => '<string>',
'SecurityGroupIdList' => ['<string>', ...],
'SubnetId' => '<string>',
],
],
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which the connection resides. If none is provided, the AWS account ID is used by default.
- ConnectionInput
-
- Required: Yes
- Type: ConnectionInput structure
A
ConnectionInputobject that redefines the connection in question. - Name
-
- Required: Yes
- Type: string
The name of the connection definition to update.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
The operation timed out.
-
The input provided was not valid.
-
An encryption operation failed.
UpdateCrawler
$result = $client->updateCrawler([/* ... */]); $promise = $client->updateCrawlerAsync([/* ... */]);
Updates a crawler. If a crawler is running, you must stop it using StopCrawler before updating it.
Parameter Syntax
$result = $client->updateCrawler([
'Classifiers' => ['<string>', ...],
'Configuration' => '<string>',
'CrawlerSecurityConfiguration' => '<string>',
'DatabaseName' => '<string>',
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
'Role' => '<string>',
'Schedule' => '<string>',
'SchemaChangePolicy' => [
'DeleteBehavior' => 'LOG|DELETE_FROM_DATABASE|DEPRECATE_IN_DATABASE',
'UpdateBehavior' => 'LOG|UPDATE_IN_DATABASE',
],
'TablePrefix' => '<string>',
'Targets' => [
'CatalogTargets' => [
[
'DatabaseName' => '<string>', // REQUIRED
'Tables' => ['<string>', ...], // REQUIRED
],
// ...
],
'DynamoDBTargets' => [
[
'Path' => '<string>',
],
// ...
],
'JdbcTargets' => [
[
'ConnectionName' => '<string>',
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
'S3Targets' => [
[
'Exclusions' => ['<string>', ...],
'Path' => '<string>',
],
// ...
],
],
]);
Parameter Details
Members
- Classifiers
-
- Type: Array of strings
A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.
- Configuration
-
- Type: string
The crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.
- CrawlerSecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used by this crawler. - DatabaseName
-
- Type: string
The AWS Glue database where results are stored, such as:
arn:aws:daylight:us-east-1::database/sometable/*. - Description
-
- Type: string
A description of the new crawler.
- Name
-
- Required: Yes
- Type: string
Name of the new crawler.
- Role
-
- Type: string
The IAM role or Amazon Resource Name (ARN) of an IAM role that is used by the new crawler to access customer resources.
- Schedule
-
- Type: string
A
cronexpression used to specify the schedule. For more information, see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, specifycron(15 12 * * ? *). - SchemaChangePolicy
-
- Type: SchemaChangePolicy structure
The policy for the crawler's update and deletion behavior.
- TablePrefix
-
- Type: string
The table prefix used for catalog tables that are created.
- Targets
-
- Type: CrawlerTargets structure
A list of targets to crawl.
Result Syntax
[]
Result Details
Errors
-
The input provided was not valid.
-
There was a version conflict.
-
A specified entity does not exist
-
The operation cannot be performed because the crawler is already running.
-
The operation timed out.
UpdateCrawlerSchedule
$result = $client->updateCrawlerSchedule([/* ... */]); $promise = $client->updateCrawlerScheduleAsync([/* ... */]);
Updates the schedule of a crawler using a cron expression.
Parameter Syntax
$result = $client->updateCrawlerSchedule([
'CrawlerName' => '<string>', // REQUIRED
'Schedule' => '<string>',
]);
Parameter Details
Members
- CrawlerName
-
- Required: Yes
- Type: string
The name of the crawler whose schedule to update.
- Schedule
-
- Type: string
The updated
cronexpression used to specify the schedule. For more information, see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, specifycron(15 12 * * ? *).
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
There was a version conflict.
-
SchedulerTransitioningException:
The specified scheduler is transitioning.
-
The operation timed out.
UpdateDatabase
$result = $client->updateDatabase([/* ... */]); $promise = $client->updateDatabaseAsync([/* ... */]);
Updates an existing database definition in a Data Catalog.
Parameter Syntax
$result = $client->updateDatabase([
'CatalogId' => '<string>',
'DatabaseInput' => [ // REQUIRED
'Description' => '<string>',
'LocationUri' => '<string>',
'Name' => '<string>', // REQUIRED
'Parameters' => ['<string>', ...],
],
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog in which the metadata database resides. If none is supplied, the AWS account ID is used by default.
- DatabaseInput
-
- Required: Yes
- Type: DatabaseInput structure
A
DatabaseInputobject specifying the new definition of the metadata database in the catalog. - Name
-
- Required: Yes
- Type: string
The name of the database to update in the catalog. For Hive compatibility, this is folded to lowercase.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
UpdateDevEndpoint
$result = $client->updateDevEndpoint([/* ... */]); $promise = $client->updateDevEndpointAsync([/* ... */]);
Updates a specified DevEndpoint.
Parameter Syntax
$result = $client->updateDevEndpoint([
'AddArguments' => ['<string>', ...],
'AddPublicKeys' => ['<string>', ...],
'CustomLibraries' => [
'ExtraJarsS3Path' => '<string>',
'ExtraPythonLibsS3Path' => '<string>',
],
'DeleteArguments' => ['<string>', ...],
'DeletePublicKeys' => ['<string>', ...],
'EndpointName' => '<string>', // REQUIRED
'PublicKey' => '<string>',
'UpdateEtlLibraries' => true || false,
]);
Parameter Details
Members
- AddArguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The map of arguments to add the map of arguments used to configure the DevEndpoint.
- AddPublicKeys
-
- Type: Array of strings
The list of public keys for the DevEndpoint to use.
- CustomLibraries
-
- Type: DevEndpointCustomLibraries structure
Custom Python or Java libraries to be loaded in the DevEndpoint.
- DeleteArguments
-
- Type: Array of strings
The list of argument keys to be deleted from the map of arguments used to configure the DevEndpoint.
- DeletePublicKeys
-
- Type: Array of strings
The list of public keys to be deleted from the DevEndpoint.
- EndpointName
-
- Required: Yes
- Type: string
The name of the DevEndpoint to be updated.
- PublicKey
-
- Type: string
The public key for the DevEndpoint to use.
- UpdateEtlLibraries
-
- Type: boolean
True if the list of custom libraries to be loaded in the development endpoint needs to be updated, or False otherwise.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
The input provided was not valid.
-
A value could not be validated.
UpdateJob
$result = $client->updateJob([/* ... */]); $promise = $client->updateJobAsync([/* ... */]);
Updates an existing job definition.
Parameter Syntax
$result = $client->updateJob([
'JobName' => '<string>', // REQUIRED
'JobUpdate' => [ // REQUIRED
'AllocatedCapacity' => <integer>,
'Command' => [
'Name' => '<string>',
'PythonVersion' => '<string>',
'ScriptLocation' => '<string>',
],
'Connections' => [
'Connections' => ['<string>', ...],
],
'DefaultArguments' => ['<string>', ...],
'Description' => '<string>',
'ExecutionProperty' => [
'MaxConcurrentRuns' => <integer>,
],
'LogUri' => '<string>',
'MaxCapacity' => <float>,
'MaxRetries' => <integer>,
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'NumberOfWorkers' => <integer>,
'Role' => '<string>',
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
'WorkerType' => 'Standard|G.1X|G.2X',
],
]);
Parameter Details
Members
- JobName
-
- Required: Yes
- Type: string
The name of the job definition to update.
- JobUpdate
-
- Required: Yes
- Type: JobUpdate structure
Specifies the values with which to update the job definition.
Result Syntax
[
'JobName' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
UpdatePartition
$result = $client->updatePartition([/* ... */]); $promise = $client->updatePartitionAsync([/* ... */]);
Updates a partition.
Parameter Syntax
$result = $client->updatePartition([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'PartitionInput' => [ // REQUIRED
'LastAccessTime' => <integer || string || DateTime>,
'LastAnalyzedTime' => <integer || string || DateTime>,
'Parameters' => ['<string>', ...],
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>', // REQUIRED
'SortOrder' => <integer>, // REQUIRED
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'Values' => ['<string>', ...],
],
'PartitionValueList' => ['<string>', ...], // REQUIRED
'TableName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the partition to be updated resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database in which the table in question resides.
- PartitionInput
-
- Required: Yes
- Type: PartitionInput structure
The new partition object to which to update the partition.
- PartitionValueList
-
- Required: Yes
- Type: Array of strings
A list of the values defining the partition.
- TableName
-
- Required: Yes
- Type: string
The name of the table where the partition to be updated is located.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
UpdateTable
$result = $client->updateTable([/* ... */]); $promise = $client->updateTableAsync([/* ... */]);
Updates a metadata table in the Data Catalog.
Parameter Syntax
$result = $client->updateTable([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'SkipArchive' => true || false,
'TableInput' => [ // REQUIRED
'Description' => '<string>',
'LastAccessTime' => <integer || string || DateTime>,
'LastAnalyzedTime' => <integer || string || DateTime>,
'Name' => '<string>', // REQUIRED
'Owner' => '<string>',
'Parameters' => ['<string>', ...],
'PartitionKeys' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Retention' => <integer>,
'StorageDescriptor' => [
'BucketColumns' => ['<string>', ...],
'Columns' => [
[
'Comment' => '<string>',
'Name' => '<string>', // REQUIRED
'Type' => '<string>',
],
// ...
],
'Compressed' => true || false,
'InputFormat' => '<string>',
'Location' => '<string>',
'NumberOfBuckets' => <integer>,
'OutputFormat' => '<string>',
'Parameters' => ['<string>', ...],
'SerdeInfo' => [
'Name' => '<string>',
'Parameters' => ['<string>', ...],
'SerializationLibrary' => '<string>',
],
'SkewedInfo' => [
'SkewedColumnNames' => ['<string>', ...],
'SkewedColumnValueLocationMaps' => ['<string>', ...],
'SkewedColumnValues' => ['<string>', ...],
],
'SortColumns' => [
[
'Column' => '<string>', // REQUIRED
'SortOrder' => <integer>, // REQUIRED
],
// ...
],
'StoredAsSubDirectories' => true || false,
],
'TableType' => '<string>',
'ViewExpandedText' => '<string>',
'ViewOriginalText' => '<string>',
],
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database in which the table resides. For Hive compatibility, this name is entirely lowercase.
- SkipArchive
-
- Type: boolean
By default,
UpdateTablealways creates an archived version of the table before updating it. IfskipArchiveis set to true, however,UpdateTabledoes not create the archived version. - TableInput
-
- Required: Yes
- Type: TableInput structure
An updated
TableInputobject to define the metadata table in the catalog.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
-
ResourceNumberLimitExceededException:
A resource numerical limit was exceeded.
-
An encryption operation failed.
UpdateTrigger
$result = $client->updateTrigger([/* ... */]); $promise = $client->updateTriggerAsync([/* ... */]);
Updates a trigger definition.
Parameter Syntax
$result = $client->updateTrigger([
'Name' => '<string>', // REQUIRED
'TriggerUpdate' => [ // REQUIRED
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
],
]);
Parameter Details
Members
- Name
-
- Required: Yes
- Type: string
The name of the trigger to update.
- TriggerUpdate
-
- Required: Yes
- Type: TriggerUpdate structure
The new values with which to update the trigger.
Result Syntax
[
'Trigger' => [
'Actions' => [
[
'Arguments' => ['<string>', ...],
'CrawlerName' => '<string>',
'JobName' => '<string>',
'NotificationProperty' => [
'NotifyDelayAfter' => <integer>,
],
'SecurityConfiguration' => '<string>',
'Timeout' => <integer>,
],
// ...
],
'Description' => '<string>',
'Id' => '<string>',
'Name' => '<string>',
'Predicate' => [
'Conditions' => [
[
'CrawlState' => 'RUNNING|SUCCEEDED|CANCELLED|FAILED',
'CrawlerName' => '<string>',
'JobName' => '<string>',
'LogicalOperator' => 'EQUALS',
'State' => 'STARTING|RUNNING|STOPPING|STOPPED|SUCCEEDED|FAILED|TIMEOUT',
],
// ...
],
'Logical' => 'AND|ANY',
],
'Schedule' => '<string>',
'State' => 'CREATING|CREATED|ACTIVATING|ACTIVATED|DEACTIVATING|DEACTIVATED|DELETING|UPDATING',
'Type' => 'SCHEDULED|CONDITIONAL|ON_DEMAND',
'WorkflowName' => '<string>',
],
]
Result Details
Members
- Trigger
-
- Type: Trigger structure
The resulting trigger definition.
Errors
-
The input provided was not valid.
-
An internal service error occurred.
-
A specified entity does not exist
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
UpdateUserDefinedFunction
$result = $client->updateUserDefinedFunction([/* ... */]); $promise = $client->updateUserDefinedFunctionAsync([/* ... */]);
Updates an existing function definition in the Data Catalog.
Parameter Syntax
$result = $client->updateUserDefinedFunction([
'CatalogId' => '<string>',
'DatabaseName' => '<string>', // REQUIRED
'FunctionInput' => [ // REQUIRED
'ClassName' => '<string>',
'FunctionName' => '<string>',
'OwnerName' => '<string>',
'OwnerType' => 'USER|ROLE|GROUP',
'ResourceUris' => [
[
'ResourceType' => 'JAR|FILE|ARCHIVE',
'Uri' => '<string>',
],
// ...
],
],
'FunctionName' => '<string>', // REQUIRED
]);
Parameter Details
Members
- CatalogId
-
- Type: string
The ID of the Data Catalog where the function to be updated is located. If none is supplied, the AWS account ID is used by default.
- DatabaseName
-
- Required: Yes
- Type: string
The name of the catalog database where the function to be updated is located.
- FunctionInput
-
- Required: Yes
- Type: UserDefinedFunctionInput structure
A
FunctionInputobject that re-defines the function in the Data Catalog. - FunctionName
-
- Required: Yes
- Type: string
The name of the function.
Result Syntax
[]
Result Details
Errors
-
A specified entity does not exist
-
The input provided was not valid.
-
An internal service error occurred.
-
The operation timed out.
-
An encryption operation failed.
UpdateWorkflow
$result = $client->updateWorkflow([/* ... */]); $promise = $client->updateWorkflowAsync([/* ... */]);
Updates an existing workflow.
Parameter Syntax
$result = $client->updateWorkflow([
'DefaultRunProperties' => ['<string>', ...],
'Description' => '<string>',
'Name' => '<string>', // REQUIRED
]);
Parameter Details
Members
Result Syntax
[
'Name' => '<string>',
]
Result Details
Errors
-
The input provided was not valid.
-
A specified entity does not exist
-
An internal service error occurred.
-
The operation timed out.
-
ConcurrentModificationException:
Two processes are trying to modify a resource simultaneously.
Shapes
AccessDeniedException
Description
Access to a resource was denied.
Members
Action
Description
Defines an action to be initiated by a trigger.
Members
- Arguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The job arguments used when this trigger fires. For this job run, they replace the default arguments set in the job definition itself.
You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.
For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.
- CrawlerName
-
- Type: string
The name of the crawler to be used with this action.
- JobName
-
- Type: string
The name of a job to be executed.
- NotificationProperty
-
- Type: NotificationProperty structure
Specifies configuration properties of a job run notification.
- SecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used with this action. - Timeout
-
- Type: int
The
JobRuntimeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and entersTIMEOUTstatus. The default is 2,880 minutes (48 hours). This overrides the timeout value set in the parent job.
AlreadyExistsException
Description
A resource to be created or added already exists.
Members
BatchStopJobRunError
Description
Records an error that occurred when attempting to stop a specified job run.
Members
- ErrorDetail
-
- Type: ErrorDetail structure
Specifies details about the error that was encountered.
- JobName
-
- Type: string
The name of the job definition that is used in the job run in question.
- JobRunId
-
- Type: string
The
JobRunIdof the job run in question.
BatchStopJobRunSuccessfulSubmission
Description
Records a successful request to stop a specified JobRun.
Members
CatalogEntry
Description
Specifies a table definition in the AWS Glue Data Catalog.
Members
CatalogImportStatus
Description
A structure containing migration status information.
Members
CatalogTarget
Description
Specifies an AWS Glue Data Catalog target.
Members
Classifier
Description
Classifiers are triggered during a crawl task. A classifier checks whether a given file is in a format it can handle. If it is, the classifier creates a schema in the form of a StructType object that matches that data format.
You can use the standard classifiers that AWS Glue provides, or you can write your own classifiers to best categorize your data sources and specify the appropriate schemas to use for them. A classifier can be a grok classifier, an XML classifier, a JSON classifier, or a custom CSV classifier, as specified in one of the fields in the Classifier object.
Members
- CsvClassifier
-
- Type: CsvClassifier structure
A classifier for comma-separated values (CSV).
- GrokClassifier
-
- Type: GrokClassifier structure
A classifier that uses
grok. - JsonClassifier
-
- Type: JsonClassifier structure
A classifier for JSON content.
- XMLClassifier
-
- Type: XMLClassifier structure
A classifier for XML content.
CloudWatchEncryption
Description
Specifies how CloudWatch data should be encrypted.
Members
CodeGenEdge
Description
Represents a directional edge in a directed acyclic graph (DAG).
Members
CodeGenNode
Description
Represents a node in a directed acyclic graph (DAG)
Members
- Args
-
- Required: Yes
- Type: Array of CodeGenNodeArg structures
Properties of the node, in the form of name-value pairs.
- Id
-
- Required: Yes
- Type: string
A node identifier that is unique within the node's graph.
- LineNumber
-
- Type: int
The line number of the node.
- NodeType
-
- Required: Yes
- Type: string
The type of node that this is.
CodeGenNodeArg
Description
An argument or property of a node.
Members
Column
Description
A column in a Table.
Members
ConcurrentModificationException
Description
Two processes are trying to modify a resource simultaneously.
Members
ConcurrentRunsExceededException
Description
Too many jobs are being run concurrently.
Members
Condition
Description
Defines a condition under which a trigger fires.
Members
- CrawlState
-
- Type: string
The state of the crawler to which this condition applies.
- CrawlerName
-
- Type: string
The name of the crawler to which this condition applies.
- JobName
-
- Type: string
The name of the job whose
JobRunsthis condition applies to, and on which this trigger waits. - LogicalOperator
-
- Type: string
A logical operator.
- State
-
- Type: string
The condition state. Currently, the values supported are
SUCCEEDED,STOPPED,TIMEOUT, andFAILED.
ConditionCheckFailureException
Description
A specified condition was not satisfied.
Members
Connection
Description
Defines a connection to a data source.
Members
- ConnectionProperties
-
- Type: Associative array of custom strings keys (ConnectionPropertyKey) to strings
These key-value pairs define parameters for the connection:
-
HOST- The host URI: either the fully qualified domain name (FQDN) or the IPv4 address of the database host. -
PORT- The port number, between 1024 and 65535, of the port on which the database host is listening for database connections. -
USER_NAME- The name under which to log in to the database. The value string forUSER_NAMEis "USERNAME". -
PASSWORD- A password, if one is used, for the user name. -
ENCRYPTED_PASSWORD- When you enable connection password protection by settingConnectionPasswordEncryptionin the Data Catalog encryption settings, this field stores the encrypted password. -
JDBC_DRIVER_JAR_URI- The Amazon S3 path of the JAR file that contains the JDBC driver to use. -
JDBC_DRIVER_CLASS_NAME- The class name of the JDBC driver to use. -
JDBC_ENGINE- The name of the JDBC engine to use. -
JDBC_ENGINE_VERSION- The version of the JDBC engine to use. -
CONFIG_FILES- (Reserved for future use). -
INSTANCE_ID- The instance ID to use. -
JDBC_CONNECTION_URL- The URL for the JDBC connection. -
JDBC_ENFORCE_SSL- A Boolean string (true, false) specifying whether Secure Sockets Layer (SSL) with hostname matching will be enforced for the JDBC connection on the client. The default is false.
- ConnectionType
-
- Type: string
The type of the connection. Currently, only JDBC is supported; SFTP is not supported.
- CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this connection definition was created.
- Description
-
- Type: string
The description of the connection.
- LastUpdatedBy
-
- Type: string
The user, group, or role that last updated this connection definition.
- LastUpdatedTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last time that this connection definition was updated.
- MatchCriteria
-
- Type: Array of strings
A list of criteria that can be used in selecting this connection.
- Name
-
- Type: string
The name of the connection definition.
- PhysicalConnectionRequirements
-
- Type: PhysicalConnectionRequirements structure
A map of physical connection requirements, such as virtual private cloud (VPC) and
SecurityGroup, that are needed to make this connection successfully.
ConnectionInput
Description
A structure that is used to specify a connection to create or update.
Members
- ConnectionProperties
-
- Required: Yes
- Type: Associative array of custom strings keys (ConnectionPropertyKey) to strings
These key-value pairs define parameters for the connection.
- ConnectionType
-
- Required: Yes
- Type: string
The type of the connection. Currently, only JDBC is supported; SFTP is not supported.
- Description
-
- Type: string
The description of the connection.
- MatchCriteria
-
- Type: Array of strings
A list of criteria that can be used in selecting this connection.
- Name
-
- Required: Yes
- Type: string
The name of the connection.
- PhysicalConnectionRequirements
-
- Type: PhysicalConnectionRequirements structure
A map of physical connection requirements, such as virtual private cloud (VPC) and
SecurityGroup, that are needed to successfully make this connection.
ConnectionPasswordEncryption
Description
The data structure used by the Data Catalog to encrypt the password as part of CreateConnection or UpdateConnection and store it in the ENCRYPTED_PASSWORD field in the connection properties. You can enable catalog encryption or only password encryption.
When a CreationConnection request arrives containing a password, the Data Catalog first encrypts the password using your AWS KMS key. It then encrypts the whole connection object again if catalog encryption is also enabled.
This encryption requires that you set AWS KMS key permissions to enable or restrict access on the password key according to your security requirements. For example, you might want only admin users to have decrypt permission on the password key.
Members
- AwsKmsKeyId
-
- Type: string
An AWS KMS key that is used to encrypt the connection password.
If connection password protection is enabled, the caller of
CreateConnectionandUpdateConnectionneeds at leastkms:Encryptpermission on the specified AWS KMS key, to encrypt passwords before storing them in the Data Catalog.You can set the decrypt permission to enable or restrict access on the password key according to your security requirements.
- ReturnConnectionPasswordEncrypted
-
- Required: Yes
- Type: boolean
When the
ReturnConnectionPasswordEncryptedflag is set to "true", passwords remain encrypted in the responses ofGetConnectionandGetConnections. This encryption takes effect independently from catalog encryption.
ConnectionsList
Description
Specifies the connections used by a job.
Members
Crawl
Description
The details of a crawl in the workflow.
Members
- CompletedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time on which the crawl completed.
- ErrorMessage
-
- Type: string
The error message associated with the crawl.
- LogGroup
-
- Type: string
The log group associated with the crawl.
- LogStream
-
- Type: string
The log stream associated with the crawl.
- StartedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time on which the crawl started.
- State
-
- Type: string
The state of the crawler.
Crawler
Description
Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.
Members
- Classifiers
-
- Type: Array of strings
A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.
- Configuration
-
- Type: string
Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.
- CrawlElapsedTime
-
- Type: long (int|float)
If the crawler is running, contains the total time elapsed since the last crawl began.
- CrawlerSecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used by this crawler. - CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that the crawler was created.
- DatabaseName
-
- Type: string
The name of the database in which the crawler's output is stored.
- Description
-
- Type: string
A description of the crawler.
- LastCrawl
-
- Type: LastCrawlInfo structure
The status of the last crawl, and potentially error information if an error occurred.
- LastUpdated
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that the crawler was last updated.
- Name
-
- Type: string
The name of the crawler.
- Role
-
- Type: string
The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.
- Schedule
-
- Type: Schedule structure
For scheduled crawlers, the schedule when the crawler runs.
- SchemaChangePolicy
-
- Type: SchemaChangePolicy structure
The policy that specifies update and delete behaviors for the crawler.
- State
-
- Type: string
Indicates whether the crawler is running, or whether a run is pending.
- TablePrefix
-
- Type: string
The prefix added to the names of tables that are created.
- Targets
-
- Type: CrawlerTargets structure
A collection of targets to crawl.
- Version
-
- Type: long (int|float)
The version of the crawler.
CrawlerMetrics
Description
Metrics for a specified crawler.
Members
- CrawlerName
-
- Type: string
The name of the crawler.
- LastRuntimeSeconds
-
- Type: double
The duration of the crawler's most recent run, in seconds.
- MedianRuntimeSeconds
-
- Type: double
The median duration of this crawler's runs, in seconds.
- StillEstimating
-
- Type: boolean
True if the crawler is still estimating how long it will take to complete this run.
- TablesCreated
-
- Type: int
The number of tables created by this crawler.
- TablesDeleted
-
- Type: int
The number of tables deleted by this crawler.
- TablesUpdated
-
- Type: int
The number of tables updated by this crawler.
- TimeLeftSeconds
-
- Type: double
The estimated time left to complete a running crawl.
CrawlerNodeDetails
Description
The details of a Crawler node present in the workflow.
Members
- Crawls
-
- Type: Array of Crawl structures
A list of crawls represented by the crawl node.
CrawlerNotRunningException
Description
The specified crawler is not running.
Members
CrawlerRunningException
Description
The operation cannot be performed because the crawler is already running.
Members
CrawlerStoppingException
Description
The specified crawler is stopping.
Members
CrawlerTargets
Description
Specifies data stores to crawl.
Members
- CatalogTargets
-
- Type: Array of CatalogTarget structures
Specifies AWS Glue Data Catalog targets.
- DynamoDBTargets
-
- Type: Array of DynamoDBTarget structures
Specifies Amazon DynamoDB targets.
- JdbcTargets
-
- Type: Array of JdbcTarget structures
Specifies JDBC targets.
- S3Targets
-
- Type: Array of S3Target structures
Specifies Amazon Simple Storage Service (Amazon S3) targets.
CreateCsvClassifierRequest
Description
Specifies a custom CSV classifier for CreateClassifier to create.
Members
- AllowSingleColumn
-
- Type: boolean
Enables the processing of files that contain only one column.
- ContainsHeader
-
- Type: string
Indicates whether the CSV file contains a header.
- Delimiter
-
- Type: string
A custom symbol to denote what separates each column entry in the row.
- DisableValueTrimming
-
- Type: boolean
Specifies not to trim values before identifying the type of column values. The default value is true.
- Header
-
- Type: Array of strings
A list of strings representing column names.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- QuoteSymbol
-
- Type: string
A custom symbol to denote what combines content into a single column value. Must be different from the column delimiter.
CreateGrokClassifierRequest
Description
Specifies a grok classifier for CreateClassifier to create.
Members
- Classification
-
- Required: Yes
- Type: string
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.
- CustomPatterns
-
- Type: string
Optional custom grok patterns used by this classifier.
- GrokPattern
-
- Required: Yes
- Type: string
The grok pattern used by this classifier.
- Name
-
- Required: Yes
- Type: string
The name of the new classifier.
CreateJsonClassifierRequest
Description
Specifies a JSON classifier for CreateClassifier to create.
Members
- JsonPath
-
- Required: Yes
- Type: string
A
JsonPathstring defining the JSON data for the classifier to classify. AWS Glue supports a subset ofJsonPath, as described in Writing JsonPath Custom Classifiers. - Name
-
- Required: Yes
- Type: string
The name of the classifier.
CreateXMLClassifierRequest
Description
Specifies an XML classifier for CreateClassifier to create.
Members
- Classification
-
- Required: Yes
- Type: string
An identifier of the data format that the classifier matches.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- RowTag
-
- Type: string
The XML tag designating the element that contains each record in an XML document being parsed. This can't identify a self-closing element (closed by
/>). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example,<row item_a="A" item_b="B"></row>is okay, but<row item_a="A" item_b="B" />is not).
CsvClassifier
Description
A classifier for custom CSV content.
Members
- AllowSingleColumn
-
- Type: boolean
Enables the processing of files that contain only one column.
- ContainsHeader
-
- Type: string
Indicates whether the CSV file contains a header.
- CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was registered.
- Delimiter
-
- Type: string
A custom symbol to denote what separates each column entry in the row.
- DisableValueTrimming
-
- Type: boolean
Specifies not to trim values before identifying the type of column values. The default value is
true. - Header
-
- Type: Array of strings
A list of strings representing column names.
- LastUpdated
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was last updated.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- QuoteSymbol
-
- Type: string
A custom symbol to denote what combines content into a single column value. It must be different from the column delimiter.
- Version
-
- Type: long (int|float)
The version of this classifier.
DataCatalogEncryptionSettings
Description
Contains configuration information for maintaining Data Catalog security.
Members
- ConnectionPasswordEncryption
-
- Type: ConnectionPasswordEncryption structure
When connection password protection is enabled, the Data Catalog uses a customer-provided key to encrypt the password as part of
CreateConnectionorUpdateConnectionand store it in theENCRYPTED_PASSWORDfield in the connection properties. You can enable catalog encryption or only password encryption. - EncryptionAtRest
-
- Type: EncryptionAtRest structure
Specifies the encryption-at-rest configuration for the Data Catalog.
Database
Description
The Database object represents a logical grouping of tables that may reside in a Hive metastore or an RDBMS.
Members
- CreateTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time at which the metadata database was created in the catalog.
- Description
-
- Type: string
Description of the database.
- LocationUri
-
- Type: string
The location of the database (for example, an HDFS path).
- Name
-
- Required: Yes
- Type: string
Name of the database. For Hive compatibility, this is folded to lowercase when it is stored.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
These key-value pairs define parameters and properties of the database.
DatabaseInput
Description
The structure used to create or update a database.
Members
- Description
-
- Type: string
Description of the database
- LocationUri
-
- Type: string
The location of the database (for example, an HDFS path).
- Name
-
- Required: Yes
- Type: string
Name of the database. For Hive compatibility, this is folded to lowercase when it is stored.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
Thes key-value pairs define parameters and properties of the database.
DevEndpoint
Description
A development endpoint where a developer can remotely debug ETL scripts.
Members
- Arguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
A map of arguments used to configure the DevEndpoint.
Note that currently, we only support "--enable-glue-datacatalog": "" as a valid argument.
- AvailabilityZone
-
- Type: string
The AWS availability zone where this DevEndpoint is located.
- CreatedTimestamp
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The point in time at which this DevEndpoint was created.
- EndpointName
-
- Type: string
The name of the DevEndpoint.
- ExtraJarsS3Path
-
- Type: string
Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.
Please note that only pure Java/Scala libraries can currently be used on a DevEndpoint.
- ExtraPythonLibsS3Path
-
- Type: string
Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.
Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.
- FailureReason
-
- Type: string
The reason for a current failure in this DevEndpoint.
- LastModifiedTimestamp
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The point in time at which this DevEndpoint was last modified.
- LastUpdateStatus
-
- Type: string
The status of the last update.
- NumberOfNodes
-
- Type: int
The number of AWS Glue Data Processing Units (DPUs) allocated to this DevEndpoint.
- PrivateAddress
-
- Type: string
A private IP address to access the DevEndpoint within a VPC, if the DevEndpoint is created within one. The PrivateAddress field is present only when you create the DevEndpoint within your virtual private cloud (VPC).
- PublicAddress
-
- Type: string
The public IP address used by this DevEndpoint. The PublicAddress field is present only when you create a non-VPC (virtual private cloud) DevEndpoint.
- PublicKey
-
- Type: string
The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility, as the recommended attribute to use is public keys.
- PublicKeys
-
- Type: Array of strings
A list of public keys to be used by the DevEndpoints for authentication. The use of this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.
If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys: call the
UpdateDevEndpointAPI with the public key content in thedeletePublicKeysattribute, and the list of new keys in theaddPublicKeysattribute. - RoleArn
-
- Type: string
The AWS ARN of the IAM role used in this DevEndpoint.
- SecurityConfiguration
-
- Type: string
The name of the SecurityConfiguration structure to be used with this DevEndpoint.
- SecurityGroupIds
-
- Type: Array of strings
A list of security group identifiers used in this DevEndpoint.
- Status
-
- Type: string
The current status of this DevEndpoint.
- SubnetId
-
- Type: string
The subnet ID for this DevEndpoint.
- VpcId
-
- Type: string
The ID of the virtual private cloud (VPC) used by this DevEndpoint.
- YarnEndpointAddress
-
- Type: string
The YARN endpoint address used by this DevEndpoint.
- ZeppelinRemoteSparkInterpreterPort
-
- Type: int
The Apache Zeppelin port for the remote Apache Spark interpreter.
DevEndpointCustomLibraries
Description
Custom libraries to be loaded into a DevEndpoint.
Members
- ExtraJarsS3Path
-
- Type: string
Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.
Please note that only pure Java/Scala libraries can currently be used on a DevEndpoint.
- ExtraPythonLibsS3Path
-
- Type: string
Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.
Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.
DynamoDBTarget
Description
Specifies an Amazon DynamoDB table to crawl.
Members
Edge
Description
An edge represents a directed connection between two AWS Glue components which are part of the workflow the edge belongs to.
Members
EncryptionAtRest
Description
Specifies the encryption-at-rest configuration for the Data Catalog.
Members
EncryptionConfiguration
Description
Specifies an encryption configuration.
Members
- CloudWatchEncryption
-
- Type: CloudWatchEncryption structure
The encryption configuration for CloudWatch.
- JobBookmarksEncryption
-
- Type: JobBookmarksEncryption structure
The encryption configuration for Job Bookmarks.
- S3Encryption
-
- Type: Array of S3Encryption structures
The encryption configuration for S3 data.
EntityNotFoundException
Description
A specified entity does not exist
Members
ErrorDetail
Description
Contains details about an error.
Members
ExecutionProperty
Description
An execution property of a job.
Members
GetConnectionsFilter
Description
Filters the connection definitions that are returned by the GetConnections API operation.
Members
GlueEncryptionException
Description
An encryption operation failed.
Members
GrokClassifier
Description
A classifier that uses grok patterns.
Members
- Classification
-
- Required: Yes
- Type: string
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.
- CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was registered.
- CustomPatterns
-
- Type: string
Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifiers.
- GrokPattern
-
- Required: Yes
- Type: string
The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifiers.
- LastUpdated
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was last updated.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- Version
-
- Type: long (int|float)
The version of this classifier.
IdempotentParameterMismatchException
Description
The same unique identifier was associated with two different records.
Members
InternalServiceException
Description
An internal service error occurred.
Members
InvalidInputException
Description
The input provided was not valid.
Members
JdbcTarget
Description
Specifies a JDBC data store to crawl.
Members
- ConnectionName
-
- Type: string
The name of the connection to use to connect to the JDBC target.
- Exclusions
-
- Type: Array of strings
A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler.
- Path
-
- Type: string
The path of the JDBC target.
Job
Description
Specifies a job definition.
Members
- AllocatedCapacity
-
- Type: int
This field is deprecated. Use
MaxCapacityinstead.The number of AWS Glue data processing units (DPUs) allocated to runs of this job. You can allocate from 2 to 100 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
- Command
-
- Type: JobCommand structure
The
JobCommandthat executes this job. - Connections
-
- Type: ConnectionsList structure
The connections used for this job.
- CreatedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time and date that this job definition was created.
- DefaultArguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The default arguments for this job, specified as name-value pairs.
You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.
For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.
- Description
-
- Type: string
A description of the job.
- ExecutionProperty
-
- Type: ExecutionProperty structure
An
ExecutionPropertyspecifying the maximum number of concurrent runs allowed for this job. - LastModifiedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last point in time when this job definition was modified.
- LogUri
-
- Type: string
This field is reserved for future use.
- MaxCapacity
-
- Type: double
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
Do not set
Max Capacityif usingWorkerTypeandNumberOfWorkers.The value that can be allocated for
MaxCapacitydepends on whether you are running a Python shell job or an Apache Spark ETL job:-
When you specify a Python shell job (
JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU. -
When you specify an Apache Spark ETL job (
JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.
- MaxRetries
-
- Type: int
The maximum number of times to retry this job after a JobRun fails.
- Name
-
- Type: string
The name you assign to this job definition.
- NotificationProperty
-
- Type: NotificationProperty structure
Specifies configuration properties of a job notification.
- NumberOfWorkers
-
- Type: int
The number of workers of a defined
workerTypethat are allocated when a job runs.The maximum number of workers you can define are 299 for
G.1X, and 149 forG.2X. - Role
-
- Type: string
The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
- SecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used with this job. - Timeout
-
- Type: int
The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters
TIMEOUTstatus. The default is 2,880 minutes (48 hours). - WorkerType
-
- Type: string
The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.
-
For the
Standardworker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker. -
For the
G.1Xworker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs. -
For the
G.2Xworker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
JobBookmarkEntry
Description
Defines a point which a job can resume processing.
Members
JobBookmarksEncryption
Description
Specifies how Job bookmark data should be encrypted.
Members
JobCommand
Description
Specifies code executed when a job is run.
Members
- Name
-
- Type: string
The name of the job command. For an Apache Spark ETL job, this must be
glueetl. For a Python shell job, it must bepythonshell. - PythonVersion
-
- Type: string
The Python version being used to execute a Python shell job. Allowed values are 2 or 3.
- ScriptLocation
-
- Type: string
Specifies the Amazon Simple Storage Service (Amazon S3) path to a script that executes a job.
JobNodeDetails
Description
The details of a Job node present in the workflow.
Members
- JobRuns
-
- Type: Array of JobRun structures
The information for the job runs represented by the job node.
JobRun
Description
Contains information about a job run.
Members
- AllocatedCapacity
-
- Type: int
This field is deprecated. Use
MaxCapacityinstead.The number of AWS Glue data processing units (DPUs) allocated to this JobRun. From 2 to 100 DPUs can be allocated; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
- Arguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The job arguments associated with this run. For this job run, they replace the default arguments set in the job definition itself.
You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
For information about how to specify and consume your own job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.
For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.
- Attempt
-
- Type: int
The number of the attempt to run this job.
- CompletedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time that this job run completed.
- ErrorMessage
-
- Type: string
An error message associated with this job run.
- ExecutionTime
-
- Type: int
The amount of time (in seconds) that the job run consumed resources.
- Id
-
- Type: string
The ID of this job run.
- JobName
-
- Type: string
The name of the job definition being used in this run.
- JobRunState
-
- Type: string
The current state of the job run.
- LastModifiedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last time that this job run was modified.
- LogGroupName
-
- Type: string
The name of the log group for secure logging that can be server-side encrypted in Amazon CloudWatch using AWS KMS. This name can be
/aws-glue/jobs/, in which case the default encryption isNONE. If you add a role name andSecurityConfigurationname (in other words,/aws-glue/jobs-yourRoleName-yourSecurityConfigurationName/), then that security configuration is used to encrypt the log group. - MaxCapacity
-
- Type: double
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
Do not set
Max Capacityif usingWorkerTypeandNumberOfWorkers.The value that can be allocated for
MaxCapacitydepends on whether you are running a Python shell job or an Apache Spark ETL job:-
When you specify a Python shell job (
JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU. -
When you specify an Apache Spark ETL job (
JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.
- NotificationProperty
-
- Type: NotificationProperty structure
Specifies configuration properties of a job run notification.
- NumberOfWorkers
-
- Type: int
The number of workers of a defined
workerTypethat are allocated when a job runs.The maximum number of workers you can define are 299 for
G.1X, and 149 forG.2X. - PredecessorRuns
-
- Type: Array of Predecessor structures
A list of predecessors to this job run.
- PreviousRunId
-
- Type: string
The ID of the previous run of this job. For example, the
JobRunIdspecified in theStartJobRunaction. - SecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used with this job run. - StartedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time at which this job run was started.
- Timeout
-
- Type: int
The
JobRuntimeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and entersTIMEOUTstatus. The default is 2,880 minutes (48 hours). This overrides the timeout value set in the parent job. - TriggerName
-
- Type: string
The name of the trigger that started this job run.
- WorkerType
-
- Type: string
The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.
-
For the
Standardworker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker. -
For the
G.1Xworker type, each worker provides 4 vCPU, 16 GB of memory and a 64GB disk, and 1 executor per worker. -
For the
G.2Xworker type, each worker provides 8 vCPU, 32 GB of memory and a 128GB disk, and 1 executor per worker.
JobUpdate
Description
Specifies information used to update an existing job definition. The previous job definition is completely overwritten by this information.
Members
- AllocatedCapacity
-
- Type: int
This field is deprecated. Use
MaxCapacityinstead.The number of AWS Glue data processing units (DPUs) to allocate to this job. You can allocate from 2 to 100 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
- Command
-
- Type: JobCommand structure
The
JobCommandthat executes this job (required). - Connections
-
- Type: ConnectionsList structure
The connections used for this job.
- DefaultArguments
-
- Type: Associative array of custom strings keys (GenericString) to strings
The default arguments for this job.
You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes.
For information about how to specify and consume your own Job arguments, see the Calling AWS Glue APIs in Python topic in the developer guide.
For information about the key-value pairs that AWS Glue consumes to set up your job, see the Special Parameters Used by AWS Glue topic in the developer guide.
- Description
-
- Type: string
Description of the job being defined.
- ExecutionProperty
-
- Type: ExecutionProperty structure
An
ExecutionPropertyspecifying the maximum number of concurrent runs allowed for this job. - LogUri
-
- Type: string
This field is reserved for future use.
- MaxCapacity
-
- Type: double
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the AWS Glue pricing page.
Do not set
Max Capacityif usingWorkerTypeandNumberOfWorkers.The value that can be allocated for
MaxCapacitydepends on whether you are running a Python shell job or an Apache Spark ETL job:-
When you specify a Python shell job (
JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU. -
When you specify an Apache Spark ETL job (
JobCommand.Name="glueetl"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.
- MaxRetries
-
- Type: int
The maximum number of times to retry this job if it fails.
- NotificationProperty
-
- Type: NotificationProperty structure
Specifies the configuration properties of a job notification.
- NumberOfWorkers
-
- Type: int
The number of workers of a defined
workerTypethat are allocated when a job runs.The maximum number of workers you can define are 299 for
G.1X, and 149 forG.2X. - Role
-
- Type: string
The name or Amazon Resource Name (ARN) of the IAM role associated with this job (required).
- SecurityConfiguration
-
- Type: string
The name of the
SecurityConfigurationstructure to be used with this job. - Timeout
-
- Type: int
The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters
TIMEOUTstatus. The default is 2,880 minutes (48 hours). - WorkerType
-
- Type: string
The type of predefined worker that is allocated when a job runs. Accepts a value of Standard, G.1X, or G.2X.
-
For the
Standardworker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker. -
For the
G.1Xworker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs. -
For the
G.2Xworker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.
JsonClassifier
Description
A classifier for JSON content.
Members
- CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was registered.
- JsonPath
-
- Required: Yes
- Type: string
A
JsonPathstring defining the JSON data for the classifier to classify. AWS Glue supports a subset ofJsonPath, as described in Writing JsonPath Custom Classifiers. - LastUpdated
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was last updated.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- Version
-
- Type: long (int|float)
The version of this classifier.
LastCrawlInfo
Description
Status and error information about the most recent crawl.
Members
- ErrorMessage
-
- Type: string
If an error occurred, the error information about the last crawl.
- LogGroup
-
- Type: string
The log group for the last crawl.
- LogStream
-
- Type: string
The log stream for the last crawl.
- MessagePrefix
-
- Type: string
The prefix for a message about this crawl.
- StartTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time at which the crawl started.
- Status
-
- Type: string
Status of the last crawl.
Location
Description
The location of resources.
Members
- DynamoDB
-
- Type: Array of CodeGenNodeArg structures
An Amazon DynamoDB table location.
- Jdbc
-
- Type: Array of CodeGenNodeArg structures
A JDBC location.
- S3
-
- Type: Array of CodeGenNodeArg structures
An Amazon Simple Storage Service (Amazon S3) location.
MappingEntry
Description
Defines a mapping.
Members
NoScheduleException
Description
There is no applicable schedule.
Members
Node
Description
A node represents an AWS Glue component like Trigger, Job etc. which is part of a workflow.
Members
- CrawlerDetails
-
- Type: CrawlerNodeDetails structure
Details of the crawler when the node represents a crawler.
- JobDetails
-
- Type: JobNodeDetails structure
Details of the Job when the node represents a Job.
- Name
-
- Type: string
The name of the AWS Glue component represented by the node.
- TriggerDetails
-
- Type: TriggerNodeDetails structure
Details of the Trigger when the node represents a Trigger.
- Type
-
- Type: string
The type of AWS Glue component represented by the node.
- UniqueId
-
- Type: string
The unique Id assigned to the node within the workflow.
NotificationProperty
Description
Specifies configuration properties of a notification.
Members
OperationTimeoutException
Order
Description
Specifies the sort order of a sorted column.
Members
Partition
Description
Represents a slice of table data.
Members
- CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time at which the partition was created.
- DatabaseName
-
- Type: string
The name of the catalog database where the table in question is located.
- LastAccessTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last time at which the partition was accessed.
- LastAnalyzedTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last time at which column statistics were computed for this partition.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
These key-value pairs define partition parameters.
- StorageDescriptor
-
- Type: StorageDescriptor structure
Provides information about the physical location where the partition is stored.
- TableName
-
- Type: string
The name of the table in question.
- Values
-
- Type: Array of strings
The values of the partition.
PartitionError
Description
Contains information about a partition error.
Members
- ErrorDetail
-
- Type: ErrorDetail structure
Details about the partition error.
- PartitionValues
-
- Type: Array of strings
The values that define the partition.
PartitionInput
Description
The structure used to create and update a partion.
Members
- LastAccessTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last time at which the partition was accessed.
- LastAnalyzedTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The last time at which column statistics were computed for this partition.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
These key-value pairs define partition parameters.
- StorageDescriptor
-
- Type: StorageDescriptor structure
Provides information about the physical location where the partition is stored.
- Values
-
- Type: Array of strings
The values of the partition. Although this parameter is not required by the SDK, you must specify this parameter for a valid input.
PartitionValueList
Description
Contains a list of values defining partitions.
Members
PhysicalConnectionRequirements
Description
Specifies the physical requirements for a connection.
Members
- AvailabilityZone
-
- Type: string
The connection's Availability Zone. This field is redundant because the specified subnet implies the Availability Zone to be used. Currently the field must be populated, but it will be deprecated in the future.
- SecurityGroupIdList
-
- Type: Array of strings
The security group ID list used by the connection.
- SubnetId
-
- Type: string
The subnet ID used by the connection.
Predecessor
Description
A job run that was used in the predicate of a conditional trigger that triggered this job run.
Members
Predicate
Description
Defines the predicate of the trigger, which determines when it fires.
Members
- Conditions
-
- Type: Array of Condition structures
A list of the conditions that determine when the trigger will fire.
- Logical
-
- Type: string
An optional field if only one condition is listed. If multiple conditions are listed, then this field is required.
ResourceNumberLimitExceededException
Description
A resource numerical limit was exceeded.
Members
ResourceUri
Description
URIs for function resources.
Members
S3Encryption
Description
Specifies how S3 data should be encrypted.
Members
S3Target
Description
Specifies a data store in Amazon Simple Storage Service (Amazon S3).
Members
- Exclusions
-
- Type: Array of strings
A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler.
- Path
-
- Type: string
The path to the Amazon S3 target.
Schedule
Description
A scheduling object using a cron statement to schedule an event.
Members
- ScheduleExpression
-
- Type: string
A
cronexpression used to specify the schedule. For more information, see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, specifycron(15 12 * * ? *). - State
-
- Type: string
The state of the schedule.
SchedulerNotRunningException
Description
The specified scheduler is not running.
Members
SchedulerRunningException
Description
The specified scheduler is already running.
Members
SchedulerTransitioningException
Description
The specified scheduler is transitioning.
Members
SchemaChangePolicy
Description
A policy that specifies update and deletion behaviors for the crawler.
Members
SecurityConfiguration
Description
Specifies a security configuration.
Members
- CreatedTimeStamp
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time at which this security configuration was created.
- EncryptionConfiguration
-
- Type: EncryptionConfiguration structure
The encryption configuration associated with this security configuration.
- Name
-
- Type: string
The name of the security configuration.
Segment
Description
Defines a non-overlapping region of a table's partitions, allowing multiple requests to be executed in parallel.
Members
SerDeInfo
Description
Information about a serialization/deserialization program (SerDe) which serves as an extractor and loader.
Members
- Name
-
- Type: string
Name of the SerDe.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
These key-value pairs define initialization parameters for the SerDe.
- SerializationLibrary
-
- Type: string
Usually the class that implements the SerDe. An example is:
org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe.
SkewedInfo
Description
Specifies skewed values in a table. Skewed are ones that occur with very high frequency.
Members
- SkewedColumnNames
-
- Type: Array of strings
A list of names of columns that contain skewed values.
- SkewedColumnValueLocationMaps
-
- Type: Associative array of custom strings keys (ColumnValuesString) to strings
A mapping of skewed values to the columns that contain them.
- SkewedColumnValues
-
- Type: Array of strings
A list of values that appear so frequently as to be considered skewed.
StorageDescriptor
Description
Describes the physical storage of table data.
Members
- BucketColumns
-
- Type: Array of strings
A list of reducer grouping columns, clustering columns, and bucketing columns in the table.
- Columns
-
- Type: Array of Column structures
A list of the
Columnsin the table. - Compressed
-
- Type: boolean
True if the data in the table is compressed, or False if not.
- InputFormat
-
- Type: string
The input format:
SequenceFileInputFormat(binary), orTextInputFormat, or a custom format. - Location
-
- Type: string
The physical location of the table. By default this takes the form of the warehouse location, followed by the database location in the warehouse, followed by the table name.
- NumberOfBuckets
-
- Type: int
Must be specified if the table contains any dimension columns.
- OutputFormat
-
- Type: string
The output format:
SequenceFileOutputFormat(binary), orIgnoreKeyTextOutputFormat, or a custom format. - Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
User-supplied properties in key-value form.
- SerdeInfo
-
- Type: SerDeInfo structure
Serialization/deserialization (SerDe) information.
- SkewedInfo
-
- Type: SkewedInfo structure
Information about values that appear very frequently in a column (skewed values).
- SortColumns
-
- Type: Array of Order structures
A list specifying the sort order of each bucket in the table.
- StoredAsSubDirectories
-
- Type: boolean
True if the table data is stored in subdirectories, or False if not.
Table
Description
Represents a collection of related data organized in columns and rows.
Members
- CreateTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
Time when the table definition was created in the Data Catalog.
- CreatedBy
-
- Type: string
Person or entity who created the table.
- DatabaseName
-
- Type: string
Name of the metadata database where the table metadata resides. For Hive compatibility, this must be all lowercase.
- Description
-
- Type: string
Description of the table.
- LastAccessTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
Last time the table was accessed. This is usually taken from HDFS, and may not be reliable.
- LastAnalyzedTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
Last time column statistics were computed for this table.
- Name
-
- Required: Yes
- Type: string
Name of the table. For Hive compatibility, this must be entirely lowercase.
- Owner
-
- Type: string
Owner of the table.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
These key-value pairs define properties associated with the table.
- PartitionKeys
-
- Type: Array of Column structures
A list of columns by which the table is partitioned. Only primitive types are supported as partition keys.
When creating a table used by Athena, and you do not specify any
partitionKeys, you must at least set the value ofpartitionKeysto an empty list. For example:"PartitionKeys": [] - Retention
-
- Type: int
Retention time for this table.
- StorageDescriptor
-
- Type: StorageDescriptor structure
A storage descriptor containing information about the physical storage of this table.
- TableType
-
- Type: string
The type of this table (
EXTERNAL_TABLE,VIRTUAL_VIEW, etc.). - UpdateTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
Last time the table was updated.
- ViewExpandedText
-
- Type: string
If the table is a view, the expanded text of the view; otherwise
null. - ViewOriginalText
-
- Type: string
If the table is a view, the original text of the view; otherwise
null.
TableError
Description
An error record for table operations.
Members
- ErrorDetail
-
- Type: ErrorDetail structure
Detail about the error.
- TableName
-
- Type: string
Name of the table. For Hive compatibility, this must be entirely lowercase.
TableInput
Description
Structure used to create or update the table.
Members
- Description
-
- Type: string
Description of the table.
- LastAccessTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
Last time the table was accessed.
- LastAnalyzedTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
Last time column statistics were computed for this table.
- Name
-
- Required: Yes
- Type: string
Name of the table. For Hive compatibility, this is folded to lowercase when it is stored.
- Owner
-
- Type: string
Owner of the table.
- Parameters
-
- Type: Associative array of custom strings keys (KeyString) to strings
These key-value pairs define properties associated with the table.
- PartitionKeys
-
- Type: Array of Column structures
A list of columns by which the table is partitioned. Only primitive types are supported as partition keys.
When creating a table used by Athena, and you do not specify any
partitionKeys, you must at least set the value ofpartitionKeysto an empty list. For example:"PartitionKeys": [] - Retention
-
- Type: int
Retention time for this table.
- StorageDescriptor
-
- Type: StorageDescriptor structure
A storage descriptor containing information about the physical storage of this table.
- TableType
-
- Type: string
The type of this table (
EXTERNAL_TABLE,VIRTUAL_VIEW, etc.). - ViewExpandedText
-
- Type: string
If the table is a view, the expanded text of the view; otherwise
null. - ViewOriginalText
-
- Type: string
If the table is a view, the original text of the view; otherwise
null.
TableVersion
Description
Specifies a version of a table.
Members
- Table
-
- Type: Table structure
The table in question
- VersionId
-
- Type: string
The ID value that identifies this table version. A
VersionIdis a string representation of an integer. Each version is incremented by 1.
TableVersionError
Description
An error record for table-version operations.
Members
- ErrorDetail
-
- Type: ErrorDetail structure
Detail about the error.
- TableName
-
- Type: string
The name of the table in question.
- VersionId
-
- Type: string
The ID value of the version in question. A
VersionIDis a string representation of an integer. Each version is incremented by 1.
Trigger
Description
Information about a specific trigger.
Members
- Actions
-
- Type: Array of Action structures
The actions initiated by this trigger.
- Description
-
- Type: string
A description of this trigger.
- Id
-
- Type: string
Reserved for future use.
- Name
-
- Type: string
The name of the trigger.
- Predicate
-
- Type: Predicate structure
The predicate of this trigger, which defines when it will fire.
- Schedule
-
- Type: string
A
cronexpression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify:cron(15 12 * * ? *). - State
-
- Type: string
The current state of the trigger.
- Type
-
- Type: string
The type of trigger that this is.
- WorkflowName
-
- Type: string
The name of the workflow associated with the trigger.
TriggerNodeDetails
Description
The details of a Trigger node present in the workflow.
Members
- Trigger
-
- Type: Trigger structure
The information of the trigger represented by the trigger node.
TriggerUpdate
Description
A structure used to provide information used to update a trigger. This object updates the previous trigger definition by overwriting it completely.
Members
- Actions
-
- Type: Array of Action structures
The actions initiated by this trigger.
- Description
-
- Type: string
A description of this trigger.
- Name
-
- Type: string
Reserved for future use.
- Predicate
-
- Type: Predicate structure
The predicate of this trigger, which defines when it will fire.
- Schedule
-
- Type: string
A
cronexpression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify:cron(15 12 * * ? *).
UpdateCsvClassifierRequest
Description
Specifies a custom CSV classifier to be updated.
Members
- AllowSingleColumn
-
- Type: boolean
Enables the processing of files that contain only one column.
- ContainsHeader
-
- Type: string
Indicates whether the CSV file contains a header.
- Delimiter
-
- Type: string
A custom symbol to denote what separates each column entry in the row.
- DisableValueTrimming
-
- Type: boolean
Specifies not to trim values before identifying the type of column values. The default value is true.
- Header
-
- Type: Array of strings
A list of strings representing column names.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- QuoteSymbol
-
- Type: string
A custom symbol to denote what combines content into a single column value. It must be different from the column delimiter.
UpdateGrokClassifierRequest
Description
Specifies a grok classifier to update when passed to UpdateClassifier.
Members
- Classification
-
- Type: string
An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.
- CustomPatterns
-
- Type: string
Optional custom grok patterns used by this classifier.
- GrokPattern
-
- Type: string
The grok pattern used by this classifier.
- Name
-
- Required: Yes
- Type: string
The name of the
GrokClassifier.
UpdateJsonClassifierRequest
Description
Specifies a JSON classifier to be updated.
Members
- JsonPath
-
- Type: string
A
JsonPathstring defining the JSON data for the classifier to classify. AWS Glue supports a subset ofJsonPath, as described in Writing JsonPath Custom Classifiers. - Name
-
- Required: Yes
- Type: string
The name of the classifier.
UpdateXMLClassifierRequest
Description
Specifies an XML classifier to be updated.
Members
- Classification
-
- Type: string
An identifier of the data format that the classifier matches.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- RowTag
-
- Type: string
The XML tag designating the element that contains each record in an XML document being parsed. This cannot identify a self-closing element (closed by
/>). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example,<row item_a="A" item_b="B"></row>is okay, but<row item_a="A" item_b="B" />is not).
UserDefinedFunction
Description
Represents the equivalent of a Hive user-defined function (UDF) definition.
Members
- ClassName
-
- Type: string
The Java class that contains the function code.
- CreateTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time at which the function was created.
- FunctionName
-
- Type: string
The name of the function.
- OwnerName
-
- Type: string
The owner of the function.
- OwnerType
-
- Type: string
The owner type.
- ResourceUris
-
- Type: Array of ResourceUri structures
The resource URIs for the function.
UserDefinedFunctionInput
Description
A structure used to create or updata a user-defined function.
Members
- ClassName
-
- Type: string
The Java class that contains the function code.
- FunctionName
-
- Type: string
The name of the function.
- OwnerName
-
- Type: string
The owner of the function.
- OwnerType
-
- Type: string
The owner type.
- ResourceUris
-
- Type: Array of ResourceUri structures
The resource URIs for the function.
ValidationException
Description
A value could not be validated.
Members
VersionMismatchException
Description
There was a version conflict.
Members
Workflow
Description
A workflow represents a flow in which AWS Glue components should be executed to complete a logical task.
Members
- CreatedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the workflow was created.
- DefaultRunProperties
-
- Type: Associative array of custom strings keys (IdString) to strings
A collection of properties to be used as part of each execution of the workflow.
- Description
-
- Type: string
A description of the workflow.
- Graph
-
- Type: WorkflowGraph structure
The graph representing all the AWS Glue components that belong to the workflow as nodes and directed connections between them as edges.
- LastModifiedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the workflow was last modified.
- LastRun
-
- Type: WorkflowRun structure
The information about the last execution of the workflow.
- Name
-
- Type: string
The name of the workflow representing the flow.
WorkflowGraph
Description
A workflow graph represents the complete workflow containing all the AWS Glue components present in the workflow and all the directed connections between them.
Members
WorkflowRun
Description
A workflow run is an execution of a workflow providing all the runtime information.
Members
- CompletedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the workflow run completed.
- Graph
-
- Type: WorkflowGraph structure
The graph representing all the AWS Glue components that belong to the workflow as nodes and directed connections between them as edges.
- Name
-
- Type: string
Name of the workflow which was executed.
- StartedOn
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The date and time when the workflow run was started.
- Statistics
-
- Type: WorkflowRunStatistics structure
The statistics of the run.
- Status
-
- Type: string
The status of the workflow run.
- WorkflowRunId
-
- Type: string
The ID of this workflow run.
- WorkflowRunProperties
-
- Type: Associative array of custom strings keys (IdString) to strings
The workflow run properties which were set during the run.
WorkflowRunStatistics
Description
Workflow run statistics provides statistics about the workflow run.
Members
- FailedActions
-
- Type: int
Total number of Actions which have failed.
- RunningActions
-
- Type: int
Total number Actions in running state.
- StoppedActions
-
- Type: int
Total number of Actions which have stopped.
- SucceededActions
-
- Type: int
Total number of Actions which have succeeded.
- TimeoutActions
-
- Type: int
Total number of Actions which timed out.
- TotalActions
-
- Type: int
Total number of Actions in the workflow run.
XMLClassifier
Description
A classifier for XML content.
Members
- Classification
-
- Required: Yes
- Type: string
An identifier of the data format that the classifier matches.
- CreationTime
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was registered.
- LastUpdated
-
- Type: timestamp (string|DateTime or anything parsable by strtotime)
The time that this classifier was last updated.
- Name
-
- Required: Yes
- Type: string
The name of the classifier.
- RowTag
-
- Type: string
The XML tag designating the element that contains each record in an XML document being parsed. This can't identify a self-closing element (closed by
/>). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example,<row item_a="A" item_b="B"></row>is okay, but<row item_a="A" item_b="B" />is not). - Version
-
- Type: long (int|float)
The version of this classifier.
