Class: Aws::Glue::Client

Inherits:

Seahorse::Client::Base

• Object
• Seahorse::Client::Base
• Aws::Glue::Client

Includes:

ClientStubs

Defined in:

gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb

Overview

An API client for Glue. To construct a client, you need to configure a :region and :credentials.

client = Aws::Glue::Client.new(
  region: region_name,
  credentials: credentials,
  # ...
)

For details on configuring region and credentials see the developer guide.

See #initialize for a full list of supported configuration options.

Instance Attribute Summary

Attributes inherited from Seahorse::Client::Base

#config, #handlers

API Operations

• #batch_create_partition(params = {}) ⇒ Types::BatchCreatePartitionResponse

  Creates one or more partitions in a batch operation.

• #batch_delete_connection(params = {}) ⇒ Types::BatchDeleteConnectionResponse

  Deletes a list of connection definitions from the Data Catalog.

• #batch_delete_partition(params = {}) ⇒ Types::BatchDeletePartitionResponse

  Deletes one or more partitions in a batch operation.

• #batch_delete_table(params = {}) ⇒ Types::BatchDeleteTableResponse

  Deletes multiple tables at once.

• #batch_delete_table_version(params = {}) ⇒ Types::BatchDeleteTableVersionResponse

  Deletes a specified batch of versions of a table.

• #batch_get_blueprints(params = {}) ⇒ Types::BatchGetBlueprintsResponse

  Retrieves information about a list of blueprints.

• #batch_get_crawlers(params = {}) ⇒ Types::BatchGetCrawlersResponse

  Returns a list of resource metadata for a given list of crawler names.

• #batch_get_custom_entity_types(params = {}) ⇒ Types::BatchGetCustomEntityTypesResponse

  Retrieves the details for the custom patterns specified by a list of names.

• #batch_get_dev_endpoints(params = {}) ⇒ Types::BatchGetDevEndpointsResponse

  Returns a list of resource metadata for a given list of development endpoint names.

• #batch_get_jobs(params = {}) ⇒ Types::BatchGetJobsResponse

  Returns a list of resource metadata for a given list of job names.

• #batch_get_partition(params = {}) ⇒ Types::BatchGetPartitionResponse

  Retrieves partitions in a batch request.

• #batch_get_triggers(params = {}) ⇒ Types::BatchGetTriggersResponse

  Returns a list of resource metadata for a given list of trigger names.

• #batch_get_workflows(params = {}) ⇒ Types::BatchGetWorkflowsResponse

  Returns a list of resource metadata for a given list of workflow names.

• #batch_stop_job_run(params = {}) ⇒ Types::BatchStopJobRunResponse

  Stops one or more job runs for a specified job definition.

• #batch_update_partition(params = {}) ⇒ Types::BatchUpdatePartitionResponse

  Updates one or more partitions in a batch operation.

• #cancel_ml_task_run(params = {}) ⇒ Types::CancelMLTaskRunResponse

  Cancels (stops) a task run.

• #cancel_statement(params = {}) ⇒ Struct

  Cancels the statement..

• #check_schema_version_validity(params = {}) ⇒ Types::CheckSchemaVersionValidityResponse

  Validates the supplied schema.

• #create_blueprint(params = {}) ⇒ Types::CreateBlueprintResponse

  Registers a blueprint with Glue.

• #create_classifier(params = {}) ⇒ Struct

  Creates a classifier in the user's account.

• #create_connection(params = {}) ⇒ Struct

  Creates a connection definition in the Data Catalog.

• #create_crawler(params = {}) ⇒ Struct

  Creates a new crawler with specified targets, role, configuration, and optional schedule.

• #create_custom_entity_type(params = {}) ⇒ Types::CreateCustomEntityTypeResponse

  Creates a custom pattern that is used to detect sensitive data across the columns and rows of your structured data.

• #create_database(params = {}) ⇒ Struct

  Creates a new database in a Data Catalog.

• #create_dev_endpoint(params = {}) ⇒ Types::CreateDevEndpointResponse

  Creates a new development endpoint.

• #create_job(params = {}) ⇒ Types::CreateJobResponse

  Creates a new job definition.

• #create_ml_transform(params = {}) ⇒ Types::CreateMLTransformResponse

  Creates an Glue machine learning transform.

• #create_partition(params = {}) ⇒ Struct

  Creates a new partition.

• #create_partition_index(params = {}) ⇒ Struct

  Creates a specified partition index in an existing table.

• #create_registry(params = {}) ⇒ Types::CreateRegistryResponse

  Creates a new registry which may be used to hold a collection of schemas.

• #create_schema(params = {}) ⇒ Types::CreateSchemaResponse

  Creates a new schema set and registers the schema definition.

• #create_script(params = {}) ⇒ Types::CreateScriptResponse

  Transforms a directed acyclic graph (DAG) into code.

• #create_security_configuration(params = {}) ⇒ Types::CreateSecurityConfigurationResponse

  Creates a new security configuration.

• #create_session(params = {}) ⇒ Types::CreateSessionResponse

  Creates a new session.

• #create_table(params = {}) ⇒ Struct

  Creates a new table definition in the Data Catalog.

• #create_trigger(params = {}) ⇒ Types::CreateTriggerResponse

  Creates a new trigger.

• #create_user_defined_function(params = {}) ⇒ Struct

  Creates a new function definition in the Data Catalog.

• #create_workflow(params = {}) ⇒ Types::CreateWorkflowResponse

  Creates a new workflow.

• #delete_blueprint(params = {}) ⇒ Types::DeleteBlueprintResponse

  Deletes an existing blueprint.

• #delete_classifier(params = {}) ⇒ Struct

  Removes a classifier from the Data Catalog.

• #delete_column_statistics_for_partition(params = {}) ⇒ Struct

  Delete the partition column statistics of a column.

• #delete_column_statistics_for_table(params = {}) ⇒ Struct

  Retrieves table statistics of columns.

• #delete_connection(params = {}) ⇒ Struct

  Deletes a connection from the Data Catalog.

• #delete_crawler(params = {}) ⇒ Struct

  Removes a specified crawler from the Glue Data Catalog, unless the crawler state is RUNNING.

• #delete_custom_entity_type(params = {}) ⇒ Types::DeleteCustomEntityTypeResponse

  Deletes a custom pattern by specifying its name.

• #delete_database(params = {}) ⇒ Struct

  Removes a specified database from a Data Catalog.

• #delete_dev_endpoint(params = {}) ⇒ Struct

  Deletes a specified development endpoint.

• #delete_job(params = {}) ⇒ Types::DeleteJobResponse

  Deletes a specified job definition.

• #delete_ml_transform(params = {}) ⇒ Types::DeleteMLTransformResponse

  Deletes an Glue machine learning transform.

• #delete_partition(params = {}) ⇒ Struct

  Deletes a specified partition.

• #delete_partition_index(params = {}) ⇒ Struct

  Deletes a specified partition index from an existing table.

• #delete_registry(params = {}) ⇒ Types::DeleteRegistryResponse

  Delete the entire registry including schema and all of its versions.

• #delete_resource_policy(params = {}) ⇒ Struct

  Deletes a specified policy.

• #delete_schema(params = {}) ⇒ Types::DeleteSchemaResponse

  Deletes the entire schema set, including the schema set and all of its versions.

• #delete_schema_versions(params = {}) ⇒ Types::DeleteSchemaVersionsResponse

  Remove versions from the specified schema.

• #delete_security_configuration(params = {}) ⇒ Struct

  Deletes a specified security configuration.

• #delete_session(params = {}) ⇒ Types::DeleteSessionResponse

  Deletes the session.

• #delete_table(params = {}) ⇒ Struct

  Removes a table definition from the Data Catalog.

• #delete_table_version(params = {}) ⇒ Struct

  Deletes a specified version of a table.

• #delete_trigger(params = {}) ⇒ Types::DeleteTriggerResponse

  Deletes a specified trigger.

• #delete_user_defined_function(params = {}) ⇒ Struct

  Deletes an existing function definition from the Data Catalog.

• #delete_workflow(params = {}) ⇒ Types::DeleteWorkflowResponse

  Deletes a workflow.

• #get_blueprint(params = {}) ⇒ Types::GetBlueprintResponse

  Retrieves the details of a blueprint.

• #get_blueprint_run(params = {}) ⇒ Types::GetBlueprintRunResponse

  Retrieves the details of a blueprint run.

• #get_blueprint_runs(params = {}) ⇒ Types::GetBlueprintRunsResponse

  Retrieves the details of blueprint runs for a specified blueprint.

• #get_catalog_import_status(params = {}) ⇒ Types::GetCatalogImportStatusResponse

  Retrieves the status of a migration operation.

• #get_classifier(params = {}) ⇒ Types::GetClassifierResponse

  Retrieve a classifier by name.

• #get_classifiers(params = {}) ⇒ Types::GetClassifiersResponse

  Lists all classifier objects in the Data Catalog.

• #get_column_statistics_for_partition(params = {}) ⇒ Types::GetColumnStatisticsForPartitionResponse

  Retrieves partition statistics of columns.

• #get_column_statistics_for_table(params = {}) ⇒ Types::GetColumnStatisticsForTableResponse

  Retrieves table statistics of columns.

• #get_connection(params = {}) ⇒ Types::GetConnectionResponse

  Retrieves a connection definition from the Data Catalog.

• #get_connections(params = {}) ⇒ Types::GetConnectionsResponse

  Retrieves a list of connection definitions from the Data Catalog.

• #get_crawler(params = {}) ⇒ Types::GetCrawlerResponse

  Retrieves metadata for a specified crawler.

• #get_crawler_metrics(params = {}) ⇒ Types::GetCrawlerMetricsResponse

  Retrieves metrics about specified crawlers.

• #get_crawlers(params = {}) ⇒ Types::GetCrawlersResponse

  Retrieves metadata for all crawlers defined in the customer account.

• #get_custom_entity_type(params = {}) ⇒ Types::GetCustomEntityTypeResponse

  Retrieves the details of a custom pattern by specifying its name.

• #get_data_catalog_encryption_settings(params = {}) ⇒ Types::GetDataCatalogEncryptionSettingsResponse

  Retrieves the security configuration for a specified catalog.

• #get_database(params = {}) ⇒ Types::GetDatabaseResponse

  Retrieves the definition of a specified database.

• #get_databases(params = {}) ⇒ Types::GetDatabasesResponse

  Retrieves all databases defined in a given Data Catalog.

• #get_dataflow_graph(params = {}) ⇒ Types::GetDataflowGraphResponse

  Transforms a Python script into a directed acyclic graph (DAG).

• #get_dev_endpoint(params = {}) ⇒ Types::GetDevEndpointResponse

  Retrieves information about a specified development endpoint.

• #get_dev_endpoints(params = {}) ⇒ Types::GetDevEndpointsResponse

  Retrieves all the development endpoints in this AWS account.

• #get_job(params = {}) ⇒ Types::GetJobResponse

  Retrieves an existing job definition.

• #get_job_bookmark(params = {}) ⇒ Types::GetJobBookmarkResponse

  Returns information on a job bookmark entry.

• #get_job_run(params = {}) ⇒ Types::GetJobRunResponse

  Retrieves the metadata for a given job run.

• #get_job_runs(params = {}) ⇒ Types::GetJobRunsResponse

  Retrieves metadata for all runs of a given job definition.

• #get_jobs(params = {}) ⇒ Types::GetJobsResponse

  Retrieves all current job definitions.

• #get_mapping(params = {}) ⇒ Types::GetMappingResponse

  Creates mappings.

• #get_ml_task_run(params = {}) ⇒ Types::GetMLTaskRunResponse

  Gets details for a specific task run on a machine learning transform.

• #get_ml_task_runs(params = {}) ⇒ Types::GetMLTaskRunsResponse

  Gets a list of runs for a machine learning transform.

• #get_ml_transform(params = {}) ⇒ Types::GetMLTransformResponse

  Gets an Glue machine learning transform artifact and all its corresponding metadata.

• #get_ml_transforms(params = {}) ⇒ Types::GetMLTransformsResponse

  Gets a sortable, filterable list of existing Glue machine learning transforms.

• #get_partition(params = {}) ⇒ Types::GetPartitionResponse

  Retrieves information about a specified partition.

• #get_partition_indexes(params = {}) ⇒ Types::GetPartitionIndexesResponse

  Retrieves the partition indexes associated with a table.

• #get_partitions(params = {}) ⇒ Types::GetPartitionsResponse

  Retrieves information about the partitions in a table.

• #get_plan(params = {}) ⇒ Types::GetPlanResponse

  Gets code to perform a specified mapping.

• #get_registry(params = {}) ⇒ Types::GetRegistryResponse

  Describes the specified registry in detail.

• #get_resource_policies(params = {}) ⇒ Types::GetResourcePoliciesResponse

  Retrieves the resource policies set on individual resources by Resource Access Manager during cross-account permission grants.

• #get_resource_policy(params = {}) ⇒ Types::GetResourcePolicyResponse

  Retrieves a specified resource policy.

• #get_schema(params = {}) ⇒ Types::GetSchemaResponse

  Describes the specified schema in detail.

• #get_schema_by_definition(params = {}) ⇒ Types::GetSchemaByDefinitionResponse

  Retrieves a schema by the SchemaDefinition.

• #get_schema_version(params = {}) ⇒ Types::GetSchemaVersionResponse

  Get the specified schema by its unique ID assigned when a version of the schema is created or registered.

• #get_schema_versions_diff(params = {}) ⇒ Types::GetSchemaVersionsDiffResponse

  Fetches the schema version difference in the specified difference type between two stored schema versions in the Schema Registry.

• #get_security_configuration(params = {}) ⇒ Types::GetSecurityConfigurationResponse

  Retrieves a specified security configuration.

• #get_security_configurations(params = {}) ⇒ Types::GetSecurityConfigurationsResponse

  Retrieves a list of all security configurations.

• #get_session(params = {}) ⇒ Types::GetSessionResponse

  Retrieves the session.

• #get_statement(params = {}) ⇒ Types::GetStatementResponse

  Retrieves the statement.

• #get_table(params = {}) ⇒ Types::GetTableResponse

  Retrieves the Table definition in a Data Catalog for a specified table.

• #get_table_version(params = {}) ⇒ Types::GetTableVersionResponse

  Retrieves a specified version of a table.

• #get_table_versions(params = {}) ⇒ Types::GetTableVersionsResponse

  Retrieves a list of strings that identify available versions of a specified table.

• #get_tables(params = {}) ⇒ Types::GetTablesResponse

  Retrieves the definitions of some or all of the tables in a given Database.

• #get_tags(params = {}) ⇒ Types::GetTagsResponse

  Retrieves a list of tags associated with a resource.

• #get_trigger(params = {}) ⇒ Types::GetTriggerResponse

  Retrieves the definition of a trigger.

• #get_triggers(params = {}) ⇒ Types::GetTriggersResponse

  Gets all the triggers associated with a job.

• #get_unfiltered_partition_metadata(params = {}) ⇒ Types::GetUnfilteredPartitionMetadataResponse

  Returns a response object which responds to the following methods:.

• #get_unfiltered_partitions_metadata(params = {}) ⇒ Types::GetUnfilteredPartitionsMetadataResponse

  The returned response is a pageable response and is Enumerable.

• #get_unfiltered_table_metadata(params = {}) ⇒ Types::GetUnfilteredTableMetadataResponse

  Returns a response object which responds to the following methods:.

• #get_user_defined_function(params = {}) ⇒ Types::GetUserDefinedFunctionResponse

  Retrieves a specified function definition from the Data Catalog.

• #get_user_defined_functions(params = {}) ⇒ Types::GetUserDefinedFunctionsResponse

  Retrieves multiple function definitions from the Data Catalog.

• #get_workflow(params = {}) ⇒ Types::GetWorkflowResponse

  Retrieves resource metadata for a workflow.

• #get_workflow_run(params = {}) ⇒ Types::GetWorkflowRunResponse

  Retrieves the metadata for a given workflow run.

• #get_workflow_run_properties(params = {}) ⇒ Types::GetWorkflowRunPropertiesResponse

  Retrieves the workflow run properties which were set during the run.

• #get_workflow_runs(params = {}) ⇒ Types::GetWorkflowRunsResponse

  Retrieves metadata for all runs of a given workflow.

• #import_catalog_to_glue(params = {}) ⇒ Struct

  Imports an existing Amazon Athena Data Catalog to Glue.

• #list_blueprints(params = {}) ⇒ Types::ListBlueprintsResponse

  Lists all the blueprint names in an account.

• #list_crawlers(params = {}) ⇒ Types::ListCrawlersResponse

  Retrieves the names of all crawler resources in this Amazon Web Services account, or the resources with the specified tag.

• #list_crawls(params = {}) ⇒ Types::ListCrawlsResponse

  Returns all the crawls of a specified crawler.

• #list_custom_entity_types(params = {}) ⇒ Types::ListCustomEntityTypesResponse

  Lists all the custom patterns that have been created.

• #list_dev_endpoints(params = {}) ⇒ Types::ListDevEndpointsResponse

  Retrieves the names of all DevEndpoint resources in this Amazon Web Services account, or the resources with the specified tag.

• #list_jobs(params = {}) ⇒ Types::ListJobsResponse

  Retrieves the names of all job resources in this Amazon Web Services account, or the resources with the specified tag.

• #list_ml_transforms(params = {}) ⇒ Types::ListMLTransformsResponse

  Retrieves a sortable, filterable list of existing Glue machine learning transforms in this Amazon Web Services account, or the resources with the specified tag.

• #list_registries(params = {}) ⇒ Types::ListRegistriesResponse

  Returns a list of registries that you have created, with minimal registry information.

• #list_schema_versions(params = {}) ⇒ Types::ListSchemaVersionsResponse

  Returns a list of schema versions that you have created, with minimal information.

• #list_schemas(params = {}) ⇒ Types::ListSchemasResponse

  Returns a list of schemas with minimal details.

• #list_sessions(params = {}) ⇒ Types::ListSessionsResponse

  Retrieve a session..

• #list_statements(params = {}) ⇒ Types::ListStatementsResponse

  Lists statements for the session.

• #list_triggers(params = {}) ⇒ Types::ListTriggersResponse

  Retrieves the names of all trigger resources in this Amazon Web Services account, or the resources with the specified tag.

• #list_workflows(params = {}) ⇒ Types::ListWorkflowsResponse

  Lists names of workflows created in the account.

• #put_data_catalog_encryption_settings(params = {}) ⇒ Struct

  Sets the security configuration for a specified catalog.

• #put_resource_policy(params = {}) ⇒ Types::PutResourcePolicyResponse

  Sets the Data Catalog resource policy for access control.

• #put_schema_version_metadata(params = {}) ⇒ Types::PutSchemaVersionMetadataResponse

  Puts the metadata key value pair for a specified schema version ID.

• #put_workflow_run_properties(params = {}) ⇒ Struct

  Puts the specified workflow run properties for the given workflow run.

• #query_schema_version_metadata(params = {}) ⇒ Types::QuerySchemaVersionMetadataResponse

  Queries for the schema version metadata information.

• #register_schema_version(params = {}) ⇒ Types::RegisterSchemaVersionResponse

  Adds a new version to the existing schema.

• #remove_schema_version_metadata(params = {}) ⇒ Types::RemoveSchemaVersionMetadataResponse

  Removes a key value pair from the schema version metadata for the specified schema version ID.

• #reset_job_bookmark(params = {}) ⇒ Types::ResetJobBookmarkResponse

  Resets a bookmark entry.

• #resume_workflow_run(params = {}) ⇒ Types::ResumeWorkflowRunResponse

  Restarts selected nodes of a previous partially completed workflow run and resumes the workflow run.

• #run_statement(params = {}) ⇒ Types::RunStatementResponse

  Executes the statement.

• #search_tables(params = {}) ⇒ Types::SearchTablesResponse

  Searches a set of tables based on properties in the table metadata as well as on the parent database.

• #start_blueprint_run(params = {}) ⇒ Types::StartBlueprintRunResponse

  Starts a new run of the specified blueprint.

• #start_crawler(params = {}) ⇒ Struct

  Starts a crawl using the specified crawler, regardless of what is scheduled.

• #start_crawler_schedule(params = {}) ⇒ Struct

  Changes the schedule state of the specified crawler to SCHEDULED, unless the crawler is already running or the schedule state is already SCHEDULED.

• #start_export_labels_task_run(params = {}) ⇒ Types::StartExportLabelsTaskRunResponse

  Begins an asynchronous task to export all labeled data for a particular transform.

• #start_import_labels_task_run(params = {}) ⇒ Types::StartImportLabelsTaskRunResponse

  Enables you to provide additional labels (examples of truth) to be used to teach the machine learning transform and improve its quality.

• #start_job_run(params = {}) ⇒ Types::StartJobRunResponse

  Starts a job run using a job definition.

• #start_ml_evaluation_task_run(params = {}) ⇒ Types::StartMLEvaluationTaskRunResponse

  Starts a task to estimate the quality of the transform.

• #start_ml_labeling_set_generation_task_run(params = {}) ⇒ Types::StartMLLabelingSetGenerationTaskRunResponse

  Starts the active learning workflow for your machine learning transform to improve the transform's quality by generating label sets and adding labels.

• #start_trigger(params = {}) ⇒ Types::StartTriggerResponse

  Starts an existing trigger.

• #start_workflow_run(params = {}) ⇒ Types::StartWorkflowRunResponse

  Starts a new run of the specified workflow.

• #stop_crawler(params = {}) ⇒ Struct

  If the specified crawler is running, stops the crawl.

• #stop_crawler_schedule(params = {}) ⇒ Struct

  Sets the schedule state of the specified crawler to NOT_SCHEDULED, but does not stop the crawler if it is already running.

• #stop_session(params = {}) ⇒ Types::StopSessionResponse

  Stops the session.

• #stop_trigger(params = {}) ⇒ Types::StopTriggerResponse

  Stops a specified trigger.

• #stop_workflow_run(params = {}) ⇒ Struct

  Stops the execution of the specified workflow run.

• #tag_resource(params = {}) ⇒ Struct

  Adds tags to a resource.

• #untag_resource(params = {}) ⇒ Struct

  Removes tags from a resource.

• #update_blueprint(params = {}) ⇒ Types::UpdateBlueprintResponse

  Updates a registered blueprint.

• #update_classifier(params = {}) ⇒ Struct

  Modifies an existing classifier (a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field is present).

• #update_column_statistics_for_partition(params = {}) ⇒ Types::UpdateColumnStatisticsForPartitionResponse

  Creates or updates partition statistics of columns.

• #update_column_statistics_for_table(params = {}) ⇒ Types::UpdateColumnStatisticsForTableResponse

  Creates or updates table statistics of columns.

• #update_connection(params = {}) ⇒ Struct

  Updates a connection definition in the Data Catalog.

• #update_crawler(params = {}) ⇒ Struct

  Updates a crawler.

• #update_crawler_schedule(params = {}) ⇒ Struct

  Updates the schedule of a crawler using a cron expression.

• #update_database(params = {}) ⇒ Struct

  Updates an existing database definition in a Data Catalog.

• #update_dev_endpoint(params = {}) ⇒ Struct

  Updates a specified development endpoint.

• #update_job(params = {}) ⇒ Types::UpdateJobResponse

  Updates an existing job definition.

• #update_ml_transform(params = {}) ⇒ Types::UpdateMLTransformResponse

  Updates an existing machine learning transform.

• #update_partition(params = {}) ⇒ Struct

  Updates a partition.

• #update_registry(params = {}) ⇒ Types::UpdateRegistryResponse

  Updates an existing registry which is used to hold a collection of schemas.

• #update_schema(params = {}) ⇒ Types::UpdateSchemaResponse

  Updates the description, compatibility setting, or version checkpoint for a schema set.

• #update_table(params = {}) ⇒ Struct

  Updates a metadata table in the Data Catalog.

• #update_trigger(params = {}) ⇒ Types::UpdateTriggerResponse

  Updates a trigger definition.

• #update_user_defined_function(params = {}) ⇒ Struct

  Updates an existing function definition in the Data Catalog.

• #update_workflow(params = {}) ⇒ Types::UpdateWorkflowResponse

  Updates an existing workflow.

Instance Method Summary

• #initialize(options) ⇒ Client constructor

  A new instance of Client.

Methods included from ClientStubs

#api_requests, #stub_data, #stub_responses

Methods inherited from Seahorse::Client::Base

add_plugin, api, clear_plugins, define, new, #operation_names, plugins, remove_plugin, set_api, set_plugins

Methods included from Seahorse::Client::HandlerBuilder

#handle, #handle_request, #handle_response

Constructor Details

#initialize(options) ⇒ Client

Returns a new instance of Client.

Parameters:

• options (Hash)

Options Hash (options):

• :credentials (required, Aws::CredentialProvider) —

  Your AWS credentials. This can be an instance of any one of the following classes:

  • Aws::Credentials - Used for configuring static, non-refreshing credentials.

  • Aws::SharedCredentials - Used for loading static credentials from a shared file, such as ~/.aws/config.

  • Aws::AssumeRoleCredentials - Used when you need to assume a role.

  • Aws::AssumeRoleWebIdentityCredentials - Used when you need to assume a role after providing credentials via the web.

  • Aws::SSOCredentials - Used for loading credentials from AWS SSO using an access token generated from aws login.

  • Aws::ProcessCredentials - Used for loading credentials from a process that outputs to stdout.

  • Aws::InstanceProfileCredentials - Used for loading credentials from an EC2 IMDS on an EC2 instance.

  • Aws::ECSCredentials - Used for loading credentials from instances running in ECS.

  • Aws::CognitoIdentityCredentials - Used for loading credentials from the Cognito Identity service.

  When :credentials are not configured directly, the following locations will be searched for credentials:

  • Aws.config[:credentials]
  • The :access_key_id, :secret_access_key, and :session_token options.
  • ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']
  • ~/.aws/credentials
  • ~/.aws/config
  • EC2/ECS IMDS instance profile - When used by default, the timeouts are very aggressive. Construct and pass an instance of Aws::InstanceProfileCredentails or Aws::ECSCredentials to enable retries and extended timeouts. Instance profile credential fetching can be disabled by setting ENV['AWS_EC2_METADATA_DISABLED'] to true.
• :region (required, String) —

  The AWS region to connect to. The configured :region is used to determine the service :endpoint. When not passed, a default :region is searched for in the following locations:

  • Aws.config[:region]
  • ENV['AWS_REGION']
  • ENV['AMAZON_REGION']
  • ENV['AWS_DEFAULT_REGION']
  • ~/.aws/credentials
  • ~/.aws/config
• :access_key_id (String)
• :active_endpoint_cache (Boolean) — default: false —

  When set to true, a thread polling for endpoints will be running in the background every 60 secs (default). Defaults to false.

• :adaptive_retry_wait_to_fill (Boolean) — default: true —

  Used only in adaptive retry mode. When true, the request will sleep until there is sufficent client side capacity to retry the request. When false, the request will raise a RetryCapacityNotAvailableError and will not retry instead of sleeping.

• :client_side_monitoring (Boolean) — default: false —

  When true, client-side metrics will be collected for all API requests from this client.

• :client_side_monitoring_client_id (String) — default: "" —

  Allows you to provide an identifier for this client which will be attached to all generated client side metrics. Defaults to an empty string.

• :client_side_monitoring_host (String) — default: "127.0.0.1" —

  Allows you to specify the DNS hostname or IPv4 or IPv6 address that the client side monitoring agent is running on, where client metrics will be published via UDP.

• :client_side_monitoring_port (Integer) — default: 31000 —

  Required for publishing client metrics. The port that the client side monitoring agent is running on, where client metrics will be published via UDP.

• :client_side_monitoring_publisher (Aws::ClientSideMonitoring::Publisher) — default: Aws::ClientSideMonitoring::Publisher —

  Allows you to provide a custom client-side monitoring publisher class. By default, will use the Client Side Monitoring Agent Publisher.

• :convert_params (Boolean) — default: true —

  When true, an attempt is made to coerce request parameters into the required types.

• :correct_clock_skew (Boolean) — default: true —

  Used only in standard and adaptive retry modes. Specifies whether to apply a clock skew correction and retry requests with skewed client clocks.

• :defaults_mode (String) — default: "legacy" —

  See DefaultsModeConfiguration for a list of the accepted modes and the configuration defaults that are included.

• :disable_host_prefix_injection (Boolean) — default: false —

  Set to true to disable SDK automatically adding host prefix to default service endpoint when available.

• :endpoint (String) —

  The client endpoint is normally constructed from the :region option. You should only configure an :endpoint when connecting to test or custom endpoints. This should be a valid HTTP(S) URI.

• :endpoint_cache_max_entries (Integer) — default: 1000 —

  Used for the maximum size limit of the LRU cache storing endpoints data for endpoint discovery enabled operations. Defaults to 1000.

• :endpoint_cache_max_threads (Integer) — default: 10 —

  Used for the maximum threads in use for polling endpoints to be cached, defaults to 10.

• :endpoint_cache_poll_interval (Integer) — default: 60 —

  When :endpoint_discovery and :active_endpoint_cache is enabled, Use this option to config the time interval in seconds for making requests fetching endpoints information. Defaults to 60 sec.

• :endpoint_discovery (Boolean) — default: false —

  When set to true, endpoint discovery will be enabled for operations when available.

• :log_formatter (Aws::Log::Formatter) — default: Aws::Log::Formatter.default —

  The log formatter.

• :log_level (Symbol) — default: :info —

  The log level to send messages to the :logger at.

• :logger (Logger) —

  The Logger instance to send log messages to. If this option is not set, logging will be disabled.

• :max_attempts (Integer) — default: 3 —

  An integer representing the maximum number attempts that will be made for a single request, including the initial attempt. For example, setting this value to 5 will result in a request being retried up to 4 times. Used in standard and adaptive retry modes.

• :profile (String) — default: "default" —

  Used when loading credentials from the shared credentials file at HOME/.aws/credentials. When not specified, 'default' is used.

• :retry_backoff (Proc) —

  A proc or lambda used for backoff. Defaults to 2**retries * retry_base_delay. This option is only used in the legacy retry mode.

• :retry_base_delay (Float) — default: 0.3 —

  The base delay in seconds used by the default backoff function. This option is only used in the legacy retry mode.

• :retry_jitter (Symbol) — default: :none —

  A delay randomiser function used by the default backoff function. Some predefined functions can be referenced by name - :none, :equal, :full, otherwise a Proc that takes and returns a number. This option is only used in the legacy retry mode.

  @see https://www.awsarchitectureblog.com/2015/03/backoff.html

• :retry_limit (Integer) — default: 3 —

  The maximum number of times to retry failed requests. Only ~ 500 level server errors and certain ~ 400 level client errors are retried. Generally, these are throttling errors, data checksum errors, networking errors, timeout errors, auth errors, endpoint discovery, and errors from expired credentials. This option is only used in the legacy retry mode.

• :retry_max_delay (Integer) — default: 0 —

  The maximum number of seconds to delay between retries (0 for no limit) used by the default backoff function. This option is only used in the legacy retry mode.

• :retry_mode (String) — default: "legacy" —

  Specifies which retry algorithm to use. Values are:

  • legacy - The pre-existing retry behavior. This is default value if no retry mode is provided.

  • standard - A standardized set of retry rules across the AWS SDKs. This includes support for retry quotas, which limit the number of unsuccessful retries a client can make.

  • adaptive - An experimental retry mode that includes all the functionality of standard mode along with automatic client side throttling. This is a provisional mode that may change behavior in the future.

• :secret_access_key (String)
• :session_token (String)
• :simple_json (Boolean) — default: false —

  Disables request parameter conversion, validation, and formatting. Also disable response data type conversions. This option is useful when you want to ensure the highest level of performance by avoiding overhead of walking request parameters and response data structures.

  When :simple_json is enabled, the request parameters hash must be formatted exactly as the DynamoDB API expects.

• :stub_responses (Boolean) — default: false —

  Causes the client to return stubbed responses. By default fake responses are generated and returned. You can specify the response data to return or errors to raise by calling ClientStubs#stub_responses. See ClientStubs for more information.

  Please note When response stubbing is enabled, no HTTP requests are made, and retries are disabled.

• :use_dualstack_endpoint (Boolean) —

  When set to true, dualstack enabled endpoints (with .aws TLD) will be used if available.

• :use_fips_endpoint (Boolean) —

  When set to true, fips compatible endpoints will be used if available. When a fips region is used, the region is normalized and this config is set to true.

• :validate_params (Boolean) — default: true —

  When true, request parameters are validated before sending the request.

• :http_proxy (URI::HTTP, String) —

  A proxy to send requests through. Formatted like 'http://proxy.com:123'.

• :http_open_timeout (Float) — default: 15 —

  The number of seconds to wait when opening a HTTP session before raising a Timeout::Error.

• :http_read_timeout (Float) — default: 60 —

  The default number of seconds to wait for response data. This value can safely be set per-request on the session.

• :http_idle_timeout (Float) — default: 5 —

  The number of seconds a connection is allowed to sit idle before it is considered stale. Stale connections are closed and removed from the pool before making a request.

• :http_continue_timeout (Float) — default: 1 —

  The number of seconds to wait for a 100-continue response before sending the request body. This option has no effect unless the request has "Expect" header set to "100-continue". Defaults to nil which disables this behaviour. This value can safely be set per request on the session.

• :ssl_timeout (Float) — default: nil —

  Sets the SSL timeout in seconds.

• :http_wire_trace (Boolean) — default: false —

  When true, HTTP debug output will be sent to the :logger.

• :ssl_verify_peer (Boolean) — default: true —

  When true, SSL peer certificates are verified when establishing a connection.

• :ssl_ca_bundle (String) —

  Full path to the SSL certificate authority bundle file that should be used when verifying peer certificates. If you do not pass :ssl_ca_bundle or :ssl_ca_directory the the system default will be used if available.

• :ssl_ca_directory (String) —

  Full path of the directory that contains the unbundled SSL certificate authority files for verifying peer certificates. If you do not pass :ssl_ca_bundle or :ssl_ca_directory the the system default will be used if available.

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 358

def initialize(*args)
  super
end

Instance Method Details

#batch_create_partition(params = {}) ⇒ Types::BatchCreatePartitionResponse

Creates one or more partitions in a batch operation.

Examples:

Request syntax with placeholder values

resp = client.batch_create_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partition_input_list: [ # required
    {
      values: ["ValueString"],
      last_access_time: Time.now,
      storage_descriptor: {
        columns: [
          {
            name: "NameString", # required
            type: "ColumnTypeString",
            comment: "CommentString",
            parameters: {
              "KeyString" => "ParametersMapValue",
            },
          },
        ],
        location: "LocationString",
        additional_locations: ["LocationString"],
        input_format: "FormatString",
        output_format: "FormatString",
        compressed: false,
        number_of_buckets: 1,
        serde_info: {
          name: "NameString",
          serialization_library: "NameString",
          parameters: {
            "KeyString" => "ParametersMapValue",
          },
        },
        bucket_columns: ["NameString"],
        sort_columns: [
          {
            column: "NameString", # required
            sort_order: 1, # required
          },
        ],
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
        skewed_info: {
          skewed_column_names: ["NameString"],
          skewed_column_values: ["ColumnValuesString"],
          skewed_column_value_location_maps: {
            "ColumnValuesString" => "ColumnValuesString",
          },
        },
        stored_as_sub_directories: false,
        schema_reference: {
          schema_id: {
            schema_arn: "GlueResourceArn",
            schema_name: "SchemaRegistryNameString",
            registry_name: "SchemaRegistryNameString",
          },
          schema_version_id: "SchemaVersionIdString",
          schema_version_number: 1,
        },
      },
      parameters: {
        "KeyString" => "ParametersMapValue",
      },
      last_analyzed_time: Time.now,
    },
  ],
})

Response structure

resp.errors #=> Array
resp.errors[0].partition_values #=> Array
resp.errors[0].partition_values[0] #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the catalog in which the partition is to be created. Currently, this should be the Amazon Web Services account ID.

• :database_name (required, String) —

  The name of the metadata database in which the partition is to be created.

• :table_name (required, String) —

  The name of the metadata table in which the partition is to be created.

• :partition_input_list (required, Array<Types::PartitionInput>) —

  A list of PartitionInput structures that define the partitions to be created.

Returns:

• (Types::BatchCreatePartitionResponse) —

  Returns a response object which responds to the following methods:

  • #errors => Array<Types::PartitionError>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 468

def batch_create_partition(params = {}, options = {})
  req = build_request(:batch_create_partition, params)
  req.send_request(options)
end

#batch_delete_connection(params = {}) ⇒ Types::BatchDeleteConnectionResponse

Deletes a list of connection definitions from the Data Catalog.

Examples:

Request syntax with placeholder values

resp = client.batch_delete_connection({
  catalog_id: "CatalogIdString",
  connection_name_list: ["NameString"], # required
})

Response structure

resp.succeeded #=> Array
resp.succeeded[0] #=> String
resp.errors #=> Hash
resp.errors["NameString"].error_code #=> String
resp.errors["NameString"].error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog in which the connections reside. If none is provided, the Amazon Web Services account ID is used by default.

• :connection_name_list (required, Array<String>) —

  A list of names of the connections to delete.

Returns:

• (Types::BatchDeleteConnectionResponse) —

  Returns a response object which responds to the following methods:

  • #succeeded => Array<String>
  • #errors => Hash<String,Types::ErrorDetail>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 506

def batch_delete_connection(params = {}, options = {})
  req = build_request(:batch_delete_connection, params)
  req.send_request(options)
end

#batch_delete_partition(params = {}) ⇒ Types::BatchDeletePartitionResponse

Deletes one or more partitions in a batch operation.

Examples:

Request syntax with placeholder values

resp = client.batch_delete_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partitions_to_delete: [ # required
    {
      values: ["ValueString"], # required
    },
  ],
})

Response structure

resp.errors #=> Array
resp.errors[0].partition_values #=> Array
resp.errors[0].partition_values[0] #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog where the partition to be deleted resides. If none is provided, the Amazon Web Services account ID is used by default.

• :database_name (required, String) —

  The name of the catalog database in which the table in question resides.

• :table_name (required, String) —

  The name of the table that contains the partitions to be deleted.

• :partitions_to_delete (required, Array<Types::PartitionValueList>) —

  A list of PartitionInput structures that define the partitions to be deleted.

Returns:

• (Types::BatchDeletePartitionResponse) —

  Returns a response object which responds to the following methods:

  • #errors => Array<Types::PartitionError>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 558

def batch_delete_partition(params = {}, options = {})
  req = build_request(:batch_delete_partition, params)
  req.send_request(options)
end

#batch_delete_table(params = {}) ⇒ Types::BatchDeleteTableResponse

Deletes multiple tables at once.

After completing this operation, you no longer have access to the table versions and partitions that belong to the deleted table. Glue deletes these "orphaned" resources asynchronously in a timely manner, at the discretion of the service.

To ensure the immediate deletion of all related resources, before calling BatchDeleteTable, use DeleteTableVersion or BatchDeleteTableVersion, and DeletePartition or BatchDeletePartition, to delete any resources that belong to the table.

Examples:

Request syntax with placeholder values

resp = client.batch_delete_table({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  tables_to_delete: ["NameString"], # required
  transaction_id: "TransactionIdString",
})

Response structure

resp.errors #=> Array
resp.errors[0].table_name #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog where the table resides. If none is provided, the Amazon Web Services account ID is used by default.

• :database_name (required, String) —

  The name of the catalog database in which the tables to delete reside. For Hive compatibility, this name is entirely lowercase.

• :tables_to_delete (required, Array<String>) —

  A list of the table to delete.

• :transaction_id (String) —

  The transaction ID at which to delete the table contents.

Returns:

• (Types::BatchDeleteTableResponse) —

  Returns a response object which responds to the following methods:

  • #errors => Array<Types::TableError>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 616

def batch_delete_table(params = {}, options = {})
  req = build_request(:batch_delete_table, params)
  req.send_request(options)
end

#batch_delete_table_version(params = {}) ⇒ Types::BatchDeleteTableVersionResponse

Deletes a specified batch of versions of a table.

Examples:

Request syntax with placeholder values

resp = client.batch_delete_table_version({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  version_ids: ["VersionString"], # required
})

Response structure

resp.errors #=> Array
resp.errors[0].table_name #=> String
resp.errors[0].version_id #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog where the tables reside. If none is provided, the Amazon Web Services account ID is used by default.

• :database_name (required, String) —

  The database in the catalog in which the table resides. For Hive compatibility, this name is entirely lowercase.

• :table_name (required, String) —

  The name of the table. For Hive compatibility, this name is entirely lowercase.

• :version_ids (required, Array<String>) —

  A list of the IDs of versions to be deleted. A VersionId is a string representation of an integer. Each version is incremented by 1.

Returns:

• (Types::BatchDeleteTableVersionResponse) —

  Returns a response object which responds to the following methods:

  • #errors => Array<Types::TableVersionError>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 664

def batch_delete_table_version(params = {}, options = {})
  req = build_request(:batch_delete_table_version, params)
  req.send_request(options)
end

#batch_get_blueprints(params = {}) ⇒ Types::BatchGetBlueprintsResponse

Retrieves information about a list of blueprints.

Examples:

Request syntax with placeholder values

resp = client.batch_get_blueprints({
  names: ["OrchestrationNameString"], # required
  include_blueprint: false,
  include_parameter_spec: false,
})

Response structure

resp.blueprints #=> Array
resp.blueprints[0].name #=> String
resp.blueprints[0].description #=> String
resp.blueprints[0].created_on #=> Time
resp.blueprints[0].last_modified_on #=> Time
resp.blueprints[0].parameter_spec #=> String
resp.blueprints[0].blueprint_location #=> String
resp.blueprints[0].blueprint_service_location #=> String
resp.blueprints[0].status #=> String, one of "CREATING", "ACTIVE", "UPDATING", "FAILED"
resp.blueprints[0].error_message #=> String
resp.blueprints[0].last_active_definition.description #=> String
resp.blueprints[0].last_active_definition.last_modified_on #=> Time
resp.blueprints[0].last_active_definition.parameter_spec #=> String
resp.blueprints[0].last_active_definition.blueprint_location #=> String
resp.blueprints[0].last_active_definition.blueprint_service_location #=> String
resp.missing_blueprints #=> Array
resp.missing_blueprints[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :names (required, Array<String>) —

  A list of blueprint names.

• :include_blueprint (Boolean) —

  Specifies whether or not to include the blueprint in the response.

• :include_parameter_spec (Boolean) —

  Specifies whether or not to include the parameters, as a JSON string, for the blueprint in the response.

Returns:

• (Types::BatchGetBlueprintsResponse) —

  Returns a response object which responds to the following methods:

  • #blueprints => Array<Types::Blueprint>
  • #missing_blueprints => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 718

def batch_get_blueprints(params = {}, options = {})
  req = build_request(:batch_get_blueprints, params)
  req.send_request(options)
end

#batch_get_crawlers(params = {}) ⇒ Types::BatchGetCrawlersResponse

Returns a list of resource metadata for a given list of crawler names. After calling the ListCrawlers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Examples:

Request syntax with placeholder values

resp = client.batch_get_crawlers({
  crawler_names: ["NameString"], # required
})

Response structure

resp.crawlers #=> Array
resp.crawlers[0].name #=> String
resp.crawlers[0].role #=> String
resp.crawlers[0].targets.s3_targets #=> Array
resp.crawlers[0].targets.s3_targets[0].path #=> String
resp.crawlers[0].targets.s3_targets[0].exclusions #=> Array
resp.crawlers[0].targets.s3_targets[0].exclusions[0] #=> String
resp.crawlers[0].targets.s3_targets[0].connection_name #=> String
resp.crawlers[0].targets.s3_targets[0].sample_size #=> Integer
resp.crawlers[0].targets.s3_targets[0].event_queue_arn #=> String
resp.crawlers[0].targets.s3_targets[0].dlq_event_queue_arn #=> String
resp.crawlers[0].targets.jdbc_targets #=> Array
resp.crawlers[0].targets.jdbc_targets[0].connection_name #=> String
resp.crawlers[0].targets.jdbc_targets[0].path #=> String
resp.crawlers[0].targets.jdbc_targets[0].exclusions #=> Array
resp.crawlers[0].targets.jdbc_targets[0].exclusions[0] #=> String
resp.crawlers[0].targets.mongo_db_targets #=> Array
resp.crawlers[0].targets.mongo_db_targets[0].connection_name #=> String
resp.crawlers[0].targets.mongo_db_targets[0].path #=> String
resp.crawlers[0].targets.mongo_db_targets[0].scan_all #=> Boolean
resp.crawlers[0].targets.dynamo_db_targets #=> Array
resp.crawlers[0].targets.dynamo_db_targets[0].path #=> String
resp.crawlers[0].targets.dynamo_db_targets[0].scan_all #=> Boolean
resp.crawlers[0].targets.dynamo_db_targets[0].scan_rate #=> Float
resp.crawlers[0].targets.catalog_targets #=> Array
resp.crawlers[0].targets.catalog_targets[0].database_name #=> String
resp.crawlers[0].targets.catalog_targets[0].tables #=> Array
resp.crawlers[0].targets.catalog_targets[0].tables[0] #=> String
resp.crawlers[0].targets.catalog_targets[0].connection_name #=> String
resp.crawlers[0].targets.delta_targets #=> Array
resp.crawlers[0].targets.delta_targets[0].delta_tables #=> Array
resp.crawlers[0].targets.delta_targets[0].delta_tables[0] #=> String
resp.crawlers[0].targets.delta_targets[0].connection_name #=> String
resp.crawlers[0].targets.delta_targets[0].write_manifest #=> Boolean
resp.crawlers[0].database_name #=> String
resp.crawlers[0].description #=> String
resp.crawlers[0].classifiers #=> Array
resp.crawlers[0].classifiers[0] #=> String
resp.crawlers[0].recrawl_policy.recrawl_behavior #=> String, one of "CRAWL_EVERYTHING", "CRAWL_NEW_FOLDERS_ONLY", "CRAWL_EVENT_MODE"
resp.crawlers[0].schema_change_policy.update_behavior #=> String, one of "LOG", "UPDATE_IN_DATABASE"
resp.crawlers[0].schema_change_policy.delete_behavior #=> String, one of "LOG", "DELETE_FROM_DATABASE", "DEPRECATE_IN_DATABASE"
resp.crawlers[0].lineage_configuration.crawler_lineage_settings #=> String, one of "ENABLE", "DISABLE"
resp.crawlers[0].state #=> String, one of "READY", "RUNNING", "STOPPING"
resp.crawlers[0].table_prefix #=> String
resp.crawlers[0].schedule.schedule_expression #=> String
resp.crawlers[0].schedule.state #=> String, one of "SCHEDULED", "NOT_SCHEDULED", "TRANSITIONING"
resp.crawlers[0].crawl_elapsed_time #=> Integer
resp.crawlers[0].creation_time #=> Time
resp.crawlers[0].last_updated #=> Time
resp.crawlers[0].last_crawl.status #=> String, one of "SUCCEEDED", "CANCELLED", "FAILED"
resp.crawlers[0].last_crawl.error_message #=> String
resp.crawlers[0].last_crawl.log_group #=> String
resp.crawlers[0].last_crawl.log_stream #=> String
resp.crawlers[0].last_crawl.message_prefix #=> String
resp.crawlers[0].last_crawl.start_time #=> Time
resp.crawlers[0].version #=> Integer
resp.crawlers[0].configuration #=> String
resp.crawlers[0].crawler_security_configuration #=> String
resp.crawlers[0].lake_formation_configuration.use_lake_formation_credentials #=> Boolean
resp.crawlers[0].lake_formation_configuration.account_id #=> String
resp.crawlers_not_found #=> Array
resp.crawlers_not_found[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :crawler_names (required, Array<String>) —

  A list of crawler names, which might be the names returned from the ListCrawlers operation.

Returns:

• (Types::BatchGetCrawlersResponse) —

  Returns a response object which responds to the following methods:

  • #crawlers => Array<Types::Crawler>
  • #crawlers_not_found => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 813

def batch_get_crawlers(params = {}, options = {})
  req = build_request(:batch_get_crawlers, params)
  req.send_request(options)
end

#batch_get_custom_entity_types(params = {}) ⇒ Types::BatchGetCustomEntityTypesResponse

Retrieves the details for the custom patterns specified by a list of names.

Examples:

Request syntax with placeholder values

resp = client.batch_get_custom_entity_types({
  names: ["NameString"], # required
})

Response structure

resp.custom_entity_types #=> Array
resp.custom_entity_types[0].name #=> String
resp.custom_entity_types[0].regex_string #=> String
resp.custom_entity_types[0].context_words #=> Array
resp.custom_entity_types[0].context_words[0] #=> String
resp.custom_entity_types_not_found #=> Array
resp.custom_entity_types_not_found[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :names (required, Array<String>) —

  A list of names of the custom patterns that you want to retrieve.

Returns:

• (Types::BatchGetCustomEntityTypesResponse) —

  Returns a response object which responds to the following methods:

  • #custom_entity_types => Array<Types::CustomEntityType>
  • #custom_entity_types_not_found => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 849

def batch_get_custom_entity_types(params = {}, options = {})
  req = build_request(:batch_get_custom_entity_types, params)
  req.send_request(options)
end

#batch_get_dev_endpoints(params = {}) ⇒ Types::BatchGetDevEndpointsResponse

Returns a list of resource metadata for a given list of development endpoint names. After calling the ListDevEndpoints operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Examples:

Request syntax with placeholder values

resp = client.batch_get_dev_endpoints({
  dev_endpoint_names: ["GenericString"], # required
})

Response structure

resp.dev_endpoints #=> Array
resp.dev_endpoints[0].endpoint_name #=> String
resp.dev_endpoints[0].role_arn #=> String
resp.dev_endpoints[0].security_group_ids #=> Array
resp.dev_endpoints[0].security_group_ids[0] #=> String
resp.dev_endpoints[0].subnet_id #=> String
resp.dev_endpoints[0].yarn_endpoint_address #=> String
resp.dev_endpoints[0].private_address #=> String
resp.dev_endpoints[0].zeppelin_remote_spark_interpreter_port #=> Integer
resp.dev_endpoints[0].public_address #=> String
resp.dev_endpoints[0].status #=> String
resp.dev_endpoints[0].worker_type #=> String, one of "Standard", "G.1X", "G.2X"
resp.dev_endpoints[0].glue_version #=> String
resp.dev_endpoints[0].number_of_workers #=> Integer
resp.dev_endpoints[0].number_of_nodes #=> Integer
resp.dev_endpoints[0].availability_zone #=> String
resp.dev_endpoints[0].vpc_id #=> String
resp.dev_endpoints[0].extra_python_libs_s3_path #=> String
resp.dev_endpoints[0].extra_jars_s3_path #=> String
resp.dev_endpoints[0].failure_reason #=> String
resp.dev_endpoints[0].last_update_status #=> String
resp.dev_endpoints[0].created_timestamp #=> Time
resp.dev_endpoints[0].last_modified_timestamp #=> Time
resp.dev_endpoints[0].public_key #=> String
resp.dev_endpoints[0].public_keys #=> Array
resp.dev_endpoints[0].public_keys[0] #=> String
resp.dev_endpoints[0].security_configuration #=> String
resp.dev_endpoints[0].arguments #=> Hash
resp.dev_endpoints[0].arguments["GenericString"] #=> String
resp.dev_endpoints_not_found #=> Array
resp.dev_endpoints_not_found[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :dev_endpoint_names (required, Array<String>) —

  The list of DevEndpoint names, which might be the names returned from the ListDevEndpoint operation.

Returns:

• (Types::BatchGetDevEndpointsResponse) —

  Returns a response object which responds to the following methods:

  • #dev_endpoints => Array<Types::DevEndpoint>
  • #dev_endpoints_not_found => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 913

def batch_get_dev_endpoints(params = {}, options = {})
  req = build_request(:batch_get_dev_endpoints, params)
  req.send_request(options)
end

#batch_get_jobs(params = {}) ⇒ Types::BatchGetJobsResponse

Returns a list of resource metadata for a given list of job names. After calling the ListJobs operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Examples:

Request syntax with placeholder values

resp = client.batch_get_jobs({
  job_names: ["NameString"], # required
})

Response structure

resp.jobs #=> Array
resp.jobs[0].name #=> String
resp.jobs[0].description #=> String
resp.jobs[0].log_uri #=> String
resp.jobs[0].role #=> String
resp.jobs[0].created_on #=> Time
resp.jobs[0].last_modified_on #=> Time
resp.jobs[0].execution_property.max_concurrent_runs #=> Integer
resp.jobs[0].command.name #=> String
resp.jobs[0].command.script_location #=> String
resp.jobs[0].command.python_version #=> String
resp.jobs[0].default_arguments #=> Hash
resp.jobs[0].default_arguments["GenericString"] #=> String
resp.jobs[0].non_overridable_arguments #=> Hash
resp.jobs[0].non_overridable_arguments["GenericString"] #=> String
resp.jobs[0].connections.connections #=> Array
resp.jobs[0].connections.connections[0] #=> String
resp.jobs[0].max_retries #=> Integer
resp.jobs[0].allocated_capacity #=> Integer
resp.jobs[0].timeout #=> Integer
resp.jobs[0].max_capacity #=> Float
resp.jobs[0].worker_type #=> String, one of "Standard", "G.1X", "G.2X"
resp.jobs[0].number_of_workers #=> Integer
resp.jobs[0].security_configuration #=> String
resp.jobs[0].notification_property.notify_delay_after #=> Integer
resp.jobs[0].glue_version #=> String
resp.jobs[0].code_gen_configuration_nodes #=> Hash
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.connector_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.connection_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.connection_table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.schema_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].athena_connector_source.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.connector_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.connection_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.filter_predicate #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.partition_column #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.lower_bound #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.upper_bound #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.num_partitions #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.job_bookmark_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.job_bookmark_keys[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.job_bookmark_keys_sort_order #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.data_type_mapping #=> Hash
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.additional_options.data_type_mapping["JDBCDataType"] #=> String, one of "DATE", "STRING", "TIMESTAMP", "INT", "FLOAT", "LONG", "BIGDECIMAL", "BYTE", "SHORT", "DOUBLE"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.connection_table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.query #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_source.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.connector_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.connection_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.additional_options #=> Hash
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.additional_options["EnclosedInStringProperty"] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_source.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_source.redshift_tmp_dir #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_source.tmp_dir_iam_role #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_source.partition_predicate #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_source.additional_options.bounded_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_source.additional_options.bounded_files #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.paths #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.paths[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.compression_type #=> String, one of "gzip", "bzip2"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.exclusions #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.exclusions[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.group_size #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.group_files #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.recurse #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.max_band #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.max_files_in_band #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.additional_options.bounded_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.additional_options.bounded_files #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.additional_options.enable_sample_path #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.additional_options.sample_path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.separator #=> String, one of "comma", "ctrla", "pipe", "semicolon", "tab"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.escaper #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.quote_char #=> String, one of "quote", "quillemet", "single_quote", "disabled"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.multiline #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.with_header #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.write_header #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.skip_first #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.optimize_performance #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_csv_source.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.paths #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.paths[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.compression_type #=> String, one of "gzip", "bzip2"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.exclusions #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.exclusions[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.group_size #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.group_files #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.recurse #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.max_band #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.max_files_in_band #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.additional_options.bounded_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.additional_options.bounded_files #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.additional_options.enable_sample_path #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.additional_options.sample_path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.json_path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.multiline #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_json_source.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.paths #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.paths[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.compression_type #=> String, one of "snappy", "lzo", "gzip", "uncompressed", "none"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.exclusions #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.exclusions[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.group_size #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.group_files #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.recurse #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.max_band #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.max_files_in_band #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.additional_options.bounded_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.additional_options.bounded_files #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.additional_options.enable_sample_path #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.additional_options.sample_path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_parquet_source.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].relational_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].relational_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].relational_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].dynamo_db_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].dynamo_db_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].dynamo_db_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.connection_table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.connector_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.connection_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.additional_options #=> Hash
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.additional_options["EnclosedInStringProperty"] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].jdbc_connector_target.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.connector_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.connection_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.additional_options #=> Hash
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.additional_options["EnclosedInStringProperty"] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_connector_target.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.redshift_tmp_dir #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.tmp_dir_iam_role #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.upsert_redshift_options.table_location #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.upsert_redshift_options.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.upsert_redshift_options.upsert_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].redshift_target.upsert_redshift_options.upsert_keys[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.partition_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.partition_keys[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.partition_keys[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.schema_change_policy.enable_update_catalog #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_catalog_target.schema_change_policy.update_behavior #=> String, one of "UPDATE_IN_DATABASE", "LOG"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.partition_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.partition_keys[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.partition_keys[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.compression #=> String, one of "snappy", "lzo", "gzip", "uncompressed", "none"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.schema_change_policy.enable_update_catalog #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.schema_change_policy.update_behavior #=> String, one of "UPDATE_IN_DATABASE", "LOG"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.schema_change_policy.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_glue_parquet_target.schema_change_policy.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.partition_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.partition_keys[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.partition_keys[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.compression #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.format #=> String, one of "json", "csv", "avro", "orc", "parquet"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.schema_change_policy.enable_update_catalog #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.schema_change_policy.update_behavior #=> String, one of "UPDATE_IN_DATABASE", "LOG"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.schema_change_policy.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].s3_direct_target.schema_change_policy.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].to_key #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].from_path #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].from_path[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].from_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].to_type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].dropped #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].apply_mapping.mapping[0].children #=> Types::Mappings
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_fields.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_fields.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_fields.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_fields.paths #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_fields.paths[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_fields.paths[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_fields.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_fields.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_fields.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_fields.paths #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_fields.paths[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_fields.paths[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.source_path #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.source_path[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.target_path #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].rename_field.target_path[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spigot.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spigot.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spigot.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spigot.path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spigot.topk #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spigot.prob #=> Float
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.join_type #=> String, one of "equijoin", "left", "right", "outer", "leftsemi", "leftanti"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.columns[0].from #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.columns[0].keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.columns[0].keys[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].join.columns[0].keys[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].split_fields.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].split_fields.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].split_fields.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].split_fields.paths #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].split_fields.paths[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].split_fields.paths[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_from_collection.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_from_collection.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_from_collection.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].select_from_collection.index #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].fill_missing_values.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].fill_missing_values.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].fill_missing_values.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].fill_missing_values.imputed_path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].fill_missing_values.filled_path #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.logical_operator #=> String, one of "AND", "OR"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters[0].operation #=> String, one of "EQ", "LT", "GT", "LTE", "GTE", "REGEX", "ISNULL"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters[0].negated #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters[0].values #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters[0].values[0].type #=> String, one of "COLUMNEXTRACTED", "CONSTANT"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters[0].values[0].value #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].filter.filters[0].values[0].value[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.code #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.class_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].custom_code.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.sql_query #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.sql_aliases #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.sql_aliases[0].from #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.sql_aliases[0].alias #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.output_schemas #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.output_schemas[0].columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.output_schemas[0].columns[0].name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].spark_sql.output_schemas[0].columns[0].type #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.window_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.detect_schema #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.endpoint_url #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.stream_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.classification #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.delimiter #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.starting_position #=> String, one of "latest", "trim_horizon", "earliest"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.max_fetch_time_in_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.max_fetch_records_per_shard #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.max_record_per_read #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.add_idle_time_between_reads #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.idle_time_between_reads_in_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.describe_shard_interval #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.num_retries #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.retry_interval_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.max_retry_interval_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.avoid_empty_batches #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.stream_arn #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.role_arn #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.streaming_options.role_session_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.data_preview_options.polling_time #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kinesis_source.data_preview_options.record_polling_limit #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.bootstrap_servers #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.security_protocol #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.topic_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.assign #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.subscribe_pattern #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.classification #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.delimiter #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.starting_offsets #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.ending_offsets #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.poll_timeout_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.num_retries #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.retry_interval_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.max_offsets_per_trigger #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.streaming_options.min_partitions #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.window_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.detect_schema #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.data_preview_options.polling_time #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].direct_kafka_source.data_preview_options.record_polling_limit #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.window_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.detect_schema #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.endpoint_url #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.stream_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.classification #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.delimiter #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.starting_position #=> String, one of "latest", "trim_horizon", "earliest"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.max_fetch_time_in_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.max_fetch_records_per_shard #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.max_record_per_read #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.add_idle_time_between_reads #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.idle_time_between_reads_in_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.describe_shard_interval #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.num_retries #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.retry_interval_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.max_retry_interval_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.avoid_empty_batches #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.stream_arn #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.role_arn #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.streaming_options.role_session_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.data_preview_options.polling_time #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kinesis_source.data_preview_options.record_polling_limit #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.window_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.detect_schema #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.bootstrap_servers #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.security_protocol #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.connection_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.topic_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.assign #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.subscribe_pattern #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.classification #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.delimiter #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.starting_offsets #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.ending_offsets #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.poll_timeout_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.num_retries #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.retry_interval_ms #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.max_offsets_per_trigger #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.streaming_options.min_partitions #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.data_preview_options.polling_time #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].catalog_kafka_source.data_preview_options.record_polling_limit #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_check_box_list.is_empty #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_check_box_list.is_null_string #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_check_box_list.is_neg_one #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_text_list #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_text_list[0].value #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_text_list[0].datatype.id #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_null_fields.null_text_list[0].datatype.label #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.source #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.primary_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.primary_keys[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].merge.primary_keys[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].union.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].union.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].union.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].union.union_type #=> String, one of "ALL", "DISTINCT"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.pii_type #=> String, one of "RowAudit", "RowMasking", "ColumnAudit", "ColumnMasking"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.entity_types_to_detect #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.entity_types_to_detect[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.output_column_name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.sample_fraction #=> Float
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.threshold_fraction #=> Float
resp.jobs[0].code_gen_configuration_nodes["NodeId"].pii_detection.mask_value #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.groups #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.groups[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.groups[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.aggs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.aggs[0].column #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.aggs[0].column[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].aggregate.aggs[0].agg_func #=> String, one of "avg", "countDistinct", "count", "first", "last", "kurtosis", "max", "min", "skewness", "stddev_samp", "stddev_pop", "sum", "sumDistinct", "var_samp", "var_pop"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_duplicates.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_duplicates.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_duplicates.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_duplicates.columns #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_duplicates.columns[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].drop_duplicates.columns[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.partition_keys #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.partition_keys[0] #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.partition_keys[0][0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.schema_change_policy.enable_update_catalog #=> Boolean
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_target.schema_change_policy.update_behavior #=> String, one of "UPDATE_IN_DATABASE", "LOG"
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_source.partition_predicate #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_source.additional_options.bounded_size #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].governed_catalog_source.additional_options.bounded_files #=> Integer
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_source.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_source.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_source.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].microsoft_sql_server_catalog_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].my_sql_catalog_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].oracle_sql_catalog_target.table #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_target.name #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_target.inputs #=> Array
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_target.inputs[0] #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_target.database #=> String
resp.jobs[0].code_gen_configuration_nodes["NodeId"].postgre_sql_catalog_target.table #=> String
resp.jobs_not_found #=> Array
resp.jobs_not_found[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :job_names (required, Array<String>) —

  A list of job names, which might be the names returned from the ListJobs operation.

Returns:

• (Types::BatchGetJobsResponse) —

  Returns a response object which responds to the following methods:

  • #jobs => Array<Types::Job>
  • #jobs_not_found => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1454

def batch_get_jobs(params = {}, options = {})
  req = build_request(:batch_get_jobs, params)
  req.send_request(options)
end

#batch_get_partition(params = {}) ⇒ Types::BatchGetPartitionResponse

Retrieves partitions in a batch request.

Examples:

Request syntax with placeholder values

resp = client.batch_get_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partitions_to_get: [ # required
    {
      values: ["ValueString"], # required
    },
  ],
})

Response structure

resp.partitions #=> Array
resp.partitions[0].values #=> Array
resp.partitions[0].values[0] #=> String
resp.partitions[0].database_name #=> String
resp.partitions[0].table_name #=> String
resp.partitions[0].creation_time #=> Time
resp.partitions[0].last_access_time #=> Time
resp.partitions[0].storage_descriptor.columns #=> Array
resp.partitions[0].storage_descriptor.columns[0].name #=> String
resp.partitions[0].storage_descriptor.columns[0].type #=> String
resp.partitions[0].storage_descriptor.columns[0].comment #=> String
resp.partitions[0].storage_descriptor.columns[0].parameters #=> Hash
resp.partitions[0].storage_descriptor.columns[0].parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.location #=> String
resp.partitions[0].storage_descriptor.additional_locations #=> Array
resp.partitions[0].storage_descriptor.additional_locations[0] #=> String
resp.partitions[0].storage_descriptor.input_format #=> String
resp.partitions[0].storage_descriptor.output_format #=> String
resp.partitions[0].storage_descriptor.compressed #=> Boolean
resp.partitions[0].storage_descriptor.number_of_buckets #=> Integer
resp.partitions[0].storage_descriptor.serde_info.name #=> String
resp.partitions[0].storage_descriptor.serde_info.serialization_library #=> String
resp.partitions[0].storage_descriptor.serde_info.parameters #=> Hash
resp.partitions[0].storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.bucket_columns #=> Array
resp.partitions[0].storage_descriptor.bucket_columns[0] #=> String
resp.partitions[0].storage_descriptor.sort_columns #=> Array
resp.partitions[0].storage_descriptor.sort_columns[0].column #=> String
resp.partitions[0].storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.partitions[0].storage_descriptor.parameters #=> Hash
resp.partitions[0].storage_descriptor.parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.partitions[0].storage_descriptor.stored_as_sub_directories #=> Boolean
resp.partitions[0].storage_descriptor.schema_reference.schema_id.schema_arn #=> String
resp.partitions[0].storage_descriptor.schema_reference.schema_id.schema_name #=> String
resp.partitions[0].storage_descriptor.schema_reference.schema_id.registry_name #=> String
resp.partitions[0].storage_descriptor.schema_reference.schema_version_id #=> String
resp.partitions[0].storage_descriptor.schema_reference.schema_version_number #=> Integer
resp.partitions[0].parameters #=> Hash
resp.partitions[0].parameters["KeyString"] #=> String
resp.partitions[0].last_analyzed_time #=> Time
resp.partitions[0].catalog_id #=> String
resp.unprocessed_keys #=> Array
resp.unprocessed_keys[0].values #=> Array
resp.unprocessed_keys[0].values[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog where the partitions in question reside. If none is supplied, the Amazon Web Services account ID is used by default.

• :database_name (required, String) —

  The name of the catalog database where the partitions reside.

• :table_name (required, String) —

  The name of the partitions' table.

• :partitions_to_get (required, Array<Types::PartitionValueList>) —

  A list of partition values identifying the partitions to retrieve.

Returns:

• (Types::BatchGetPartitionResponse) —

  Returns a response object which responds to the following methods:

  • #partitions => Array<Types::Partition>
  • #unprocessed_keys => Array<Types::PartitionValueList>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1550

def batch_get_partition(params = {}, options = {})
  req = build_request(:batch_get_partition, params)
  req.send_request(options)
end

#batch_get_triggers(params = {}) ⇒ Types::BatchGetTriggersResponse

Returns a list of resource metadata for a given list of trigger names. After calling the ListTriggers operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Examples:

Request syntax with placeholder values

resp = client.batch_get_triggers({
  trigger_names: ["NameString"], # required
})

Response structure

resp.triggers #=> Array
resp.triggers[0].name #=> String
resp.triggers[0].workflow_name #=> String
resp.triggers[0].id #=> String
resp.triggers[0].type #=> String, one of "SCHEDULED", "CONDITIONAL", "ON_DEMAND", "EVENT"
resp.triggers[0].state #=> String, one of "CREATING", "CREATED", "ACTIVATING", "ACTIVATED", "DEACTIVATING", "DEACTIVATED", "DELETING", "UPDATING"
resp.triggers[0].description #=> String
resp.triggers[0].schedule #=> String
resp.triggers[0].actions #=> Array
resp.triggers[0].actions[0].job_name #=> String
resp.triggers[0].actions[0].arguments #=> Hash
resp.triggers[0].actions[0].arguments["GenericString"] #=> String
resp.triggers[0].actions[0].timeout #=> Integer
resp.triggers[0].actions[0].security_configuration #=> String
resp.triggers[0].actions[0].notification_property.notify_delay_after #=> Integer
resp.triggers[0].actions[0].crawler_name #=> String
resp.triggers[0].predicate.logical #=> String, one of "AND", "ANY"
resp.triggers[0].predicate.conditions #=> Array
resp.triggers[0].predicate.conditions[0].logical_operator #=> String, one of "EQUALS"
resp.triggers[0].predicate.conditions[0].job_name #=> String
resp.triggers[0].predicate.conditions[0].state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED", "TIMEOUT"
resp.triggers[0].predicate.conditions[0].crawler_name #=> String
resp.triggers[0].predicate.conditions[0].crawl_state #=> String, one of "RUNNING", "CANCELLING", "CANCELLED", "SUCCEEDED", "FAILED"
resp.triggers[0].event_batching_condition.batch_size #=> Integer
resp.triggers[0].event_batching_condition.batch_window #=> Integer
resp.triggers_not_found #=> Array
resp.triggers_not_found[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :trigger_names (required, Array<String>) —

  A list of trigger names, which may be the names returned from the ListTriggers operation.

Returns:

• (Types::BatchGetTriggersResponse) —

  Returns a response object which responds to the following methods:

  • #triggers => Array<Types::Trigger>
  • #triggers_not_found => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1610

def batch_get_triggers(params = {}, options = {})
  req = build_request(:batch_get_triggers, params)
  req.send_request(options)
end

#batch_get_workflows(params = {}) ⇒ Types::BatchGetWorkflowsResponse

Returns a list of resource metadata for a given list of workflow names. After calling the ListWorkflows operation, you can call this operation to access the data to which you have been granted permissions. This operation supports all IAM permissions, including permission conditions that uses tags.

Examples:

Request syntax with placeholder values

resp = client.batch_get_workflows({
  names: ["NameString"], # required
  include_graph: false,
})

Response structure

resp.workflows #=> Array
resp.workflows[0].name #=> String
resp.workflows[0].description #=> String
resp.workflows[0].default_run_properties #=> Hash
resp.workflows[0].default_run_properties["IdString"] #=> String
resp.workflows[0].created_on #=> Time
resp.workflows[0].last_modified_on #=> Time
resp.workflows[0].last_run.name #=> String
resp.workflows[0].last_run.workflow_run_id #=> String
resp.workflows[0].last_run.previous_run_id #=> String
resp.workflows[0].last_run.workflow_run_properties #=> Hash
resp.workflows[0].last_run.workflow_run_properties["IdString"] #=> String
resp.workflows[0].last_run.started_on #=> Time
resp.workflows[0].last_run.completed_on #=> Time
resp.workflows[0].last_run.status #=> String, one of "RUNNING", "COMPLETED", "STOPPING", "STOPPED", "ERROR"
resp.workflows[0].last_run.error_message #=> String
resp.workflows[0].last_run.statistics.total_actions #=> Integer
resp.workflows[0].last_run.statistics.timeout_actions #=> Integer
resp.workflows[0].last_run.statistics.failed_actions #=> Integer
resp.workflows[0].last_run.statistics.stopped_actions #=> Integer
resp.workflows[0].last_run.statistics.succeeded_actions #=> Integer
resp.workflows[0].last_run.statistics.running_actions #=> Integer
resp.workflows[0].last_run.graph.nodes #=> Array
resp.workflows[0].last_run.graph.nodes[0].type #=> String, one of "CRAWLER", "JOB", "TRIGGER"
resp.workflows[0].last_run.graph.nodes[0].name #=> String
resp.workflows[0].last_run.graph.nodes[0].unique_id #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.name #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.workflow_name #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.id #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.type #=> String, one of "SCHEDULED", "CONDITIONAL", "ON_DEMAND", "EVENT"
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.state #=> String, one of "CREATING", "CREATED", "ACTIVATING", "ACTIVATED", "DEACTIVATING", "DEACTIVATED", "DELETING", "UPDATING"
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.description #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.schedule #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions #=> Array
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].job_name #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].arguments #=> Hash
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].arguments["GenericString"] #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].timeout #=> Integer
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].security_configuration #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].notification_property.notify_delay_after #=> Integer
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.actions[0].crawler_name #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.logical #=> String, one of "AND", "ANY"
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.conditions #=> Array
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.conditions[0].logical_operator #=> String, one of "EQUALS"
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.conditions[0].job_name #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.conditions[0].state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED", "TIMEOUT"
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.conditions[0].crawler_name #=> String
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.predicate.conditions[0].crawl_state #=> String, one of "RUNNING", "CANCELLING", "CANCELLED", "SUCCEEDED", "FAILED"
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.event_batching_condition.batch_size #=> Integer
resp.workflows[0].last_run.graph.nodes[0].trigger_details.trigger.event_batching_condition.batch_window #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs #=> Array
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].id #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].attempt #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].previous_run_id #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].trigger_name #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].job_name #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].started_on #=> Time
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].last_modified_on #=> Time
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].completed_on #=> Time
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].job_run_state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED", "TIMEOUT"
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].arguments #=> Hash
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].arguments["GenericString"] #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].error_message #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].predecessor_runs #=> Array
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].predecessor_runs[0].job_name #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].predecessor_runs[0].run_id #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].allocated_capacity #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].execution_time #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].timeout #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].max_capacity #=> Float
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].worker_type #=> String, one of "Standard", "G.1X", "G.2X"
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].number_of_workers #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].security_configuration #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].log_group_name #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].notification_property.notify_delay_after #=> Integer
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].glue_version #=> String
resp.workflows[0].last_run.graph.nodes[0].job_details.job_runs[0].dpu_seconds #=> Float
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls #=> Array
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls[0].state #=> String, one of "RUNNING", "CANCELLING", "CANCELLED", "SUCCEEDED", "FAILED"
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls[0].started_on #=> Time
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls[0].completed_on #=> Time
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls[0].error_message #=> String
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls[0].log_group #=> String
resp.workflows[0].last_run.graph.nodes[0].crawler_details.crawls[0].log_stream #=> String
resp.workflows[0].last_run.graph.edges #=> Array
resp.workflows[0].last_run.graph.edges[0].source_id #=> String
resp.workflows[0].last_run.graph.edges[0].destination_id #=> String
resp.workflows[0].last_run.starting_event_batch_condition.batch_size #=> Integer
resp.workflows[0].last_run.starting_event_batch_condition.batch_window #=> Integer
resp.workflows[0].graph.nodes #=> Array
resp.workflows[0].graph.nodes[0].type #=> String, one of "CRAWLER", "JOB", "TRIGGER"
resp.workflows[0].graph.nodes[0].name #=> String
resp.workflows[0].graph.nodes[0].unique_id #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.name #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.workflow_name #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.id #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.type #=> String, one of "SCHEDULED", "CONDITIONAL", "ON_DEMAND", "EVENT"
resp.workflows[0].graph.nodes[0].trigger_details.trigger.state #=> String, one of "CREATING", "CREATED", "ACTIVATING", "ACTIVATED", "DEACTIVATING", "DEACTIVATED", "DELETING", "UPDATING"
resp.workflows[0].graph.nodes[0].trigger_details.trigger.description #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.schedule #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions #=> Array
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].job_name #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].arguments #=> Hash
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].arguments["GenericString"] #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].timeout #=> Integer
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].security_configuration #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].notification_property.notify_delay_after #=> Integer
resp.workflows[0].graph.nodes[0].trigger_details.trigger.actions[0].crawler_name #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.logical #=> String, one of "AND", "ANY"
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.conditions #=> Array
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.conditions[0].logical_operator #=> String, one of "EQUALS"
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.conditions[0].job_name #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.conditions[0].state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED", "TIMEOUT"
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.conditions[0].crawler_name #=> String
resp.workflows[0].graph.nodes[0].trigger_details.trigger.predicate.conditions[0].crawl_state #=> String, one of "RUNNING", "CANCELLING", "CANCELLED", "SUCCEEDED", "FAILED"
resp.workflows[0].graph.nodes[0].trigger_details.trigger.event_batching_condition.batch_size #=> Integer
resp.workflows[0].graph.nodes[0].trigger_details.trigger.event_batching_condition.batch_window #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs #=> Array
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].id #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].attempt #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].previous_run_id #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].trigger_name #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].job_name #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].started_on #=> Time
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].last_modified_on #=> Time
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].completed_on #=> Time
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].job_run_state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED", "TIMEOUT"
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].arguments #=> Hash
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].arguments["GenericString"] #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].error_message #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].predecessor_runs #=> Array
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].predecessor_runs[0].job_name #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].predecessor_runs[0].run_id #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].allocated_capacity #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].execution_time #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].timeout #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].max_capacity #=> Float
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].worker_type #=> String, one of "Standard", "G.1X", "G.2X"
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].number_of_workers #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].security_configuration #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].log_group_name #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].notification_property.notify_delay_after #=> Integer
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].glue_version #=> String
resp.workflows[0].graph.nodes[0].job_details.job_runs[0].dpu_seconds #=> Float
resp.workflows[0].graph.nodes[0].crawler_details.crawls #=> Array
resp.workflows[0].graph.nodes[0].crawler_details.crawls[0].state #=> String, one of "RUNNING", "CANCELLING", "CANCELLED", "SUCCEEDED", "FAILED"
resp.workflows[0].graph.nodes[0].crawler_details.crawls[0].started_on #=> Time
resp.workflows[0].graph.nodes[0].crawler_details.crawls[0].completed_on #=> Time
resp.workflows[0].graph.nodes[0].crawler_details.crawls[0].error_message #=> String
resp.workflows[0].graph.nodes[0].crawler_details.crawls[0].log_group #=> String
resp.workflows[0].graph.nodes[0].crawler_details.crawls[0].log_stream #=> String
resp.workflows[0].graph.edges #=> Array
resp.workflows[0].graph.edges[0].source_id #=> String
resp.workflows[0].graph.edges[0].destination_id #=> String
resp.workflows[0].max_concurrent_runs #=> Integer
resp.workflows[0].blueprint_details.blueprint_name #=> String
resp.workflows[0].blueprint_details.run_id #=> String
resp.missing_workflows #=> Array
resp.missing_workflows[0] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :names (required, Array<String>) —

  A list of workflow names, which may be the names returned from the ListWorkflows operation.

• :include_graph (Boolean) —

  Specifies whether to include a graph when returning the workflow resource metadata.

Returns:

• (Types::BatchGetWorkflowsResponse) —

  Returns a response object which responds to the following methods:

  • #workflows => Array<Types::Workflow>
  • #missing_workflows => Array<String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1807

def batch_get_workflows(params = {}, options = {})
  req = build_request(:batch_get_workflows, params)
  req.send_request(options)
end

#batch_stop_job_run(params = {}) ⇒ Types::BatchStopJobRunResponse

Stops one or more job runs for a specified job definition.

Examples:

Request syntax with placeholder values

resp = client.batch_stop_job_run({
  job_name: "NameString", # required
  job_run_ids: ["IdString"], # required
})

Response structure

resp.successful_submissions #=> Array
resp.successful_submissions[0].job_name #=> String
resp.successful_submissions[0].job_run_id #=> String
resp.errors #=> Array
resp.errors[0].job_name #=> String
resp.errors[0].job_run_id #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :job_name (required, String) —

  The name of the job definition for which to stop job runs.

• :job_run_ids (required, Array<String>) —

  A list of the JobRunIds that should be stopped for that job definition.

Returns:

• (Types::BatchStopJobRunResponse) —

  Returns a response object which responds to the following methods:

  • #successful_submissions => Array<Types::BatchStopJobRunSuccessfulSubmission>
  • #errors => Array<Types::BatchStopJobRunError>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1848

def batch_stop_job_run(params = {}, options = {})
  req = build_request(:batch_stop_job_run, params)
  req.send_request(options)
end

#batch_update_partition(params = {}) ⇒ Types::BatchUpdatePartitionResponse

Updates one or more partitions in a batch operation.

Examples:

Request syntax with placeholder values

resp = client.batch_update_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  entries: [ # required
    {
      partition_value_list: ["ValueString"], # required
      partition_input: { # required
        values: ["ValueString"],
        last_access_time: Time.now,
        storage_descriptor: {
          columns: [
            {
              name: "NameString", # required
              type: "ColumnTypeString",
              comment: "CommentString",
              parameters: {
                "KeyString" => "ParametersMapValue",
              },
            },
          ],
          location: "LocationString",
          additional_locations: ["LocationString"],
          input_format: "FormatString",
          output_format: "FormatString",
          compressed: false,
          number_of_buckets: 1,
          serde_info: {
            name: "NameString",
            serialization_library: "NameString",
            parameters: {
              "KeyString" => "ParametersMapValue",
            },
          },
          bucket_columns: ["NameString"],
          sort_columns: [
            {
              column: "NameString", # required
              sort_order: 1, # required
            },
          ],
          parameters: {
            "KeyString" => "ParametersMapValue",
          },
          skewed_info: {
            skewed_column_names: ["NameString"],
            skewed_column_values: ["ColumnValuesString"],
            skewed_column_value_location_maps: {
              "ColumnValuesString" => "ColumnValuesString",
            },
          },
          stored_as_sub_directories: false,
          schema_reference: {
            schema_id: {
              schema_arn: "GlueResourceArn",
              schema_name: "SchemaRegistryNameString",
              registry_name: "SchemaRegistryNameString",
            },
            schema_version_id: "SchemaVersionIdString",
            schema_version_number: 1,
          },
        },
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
        last_analyzed_time: Time.now,
      },
    },
  ],
})

Response structure

resp.errors #=> Array
resp.errors[0].partition_value_list #=> Array
resp.errors[0].partition_value_list[0] #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the catalog in which the partition is to be updated. Currently, this should be the Amazon Web Services account ID.

• :database_name (required, String) —

  The name of the metadata database in which the partition is to be updated.

• :table_name (required, String) —

  The name of the metadata table in which the partition is to be updated.

• :entries (required, Array<Types::BatchUpdatePartitionRequestEntry>) —

  A list of up to 100 BatchUpdatePartitionRequestEntry objects to update.

Returns:

• (Types::BatchUpdatePartitionResponse) —

  Returns a response object which responds to the following methods:

  • #errors => Array<Types::BatchUpdatePartitionFailureEntry>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1960

def batch_update_partition(params = {}, options = {})
  req = build_request(:batch_update_partition, params)
  req.send_request(options)
end

#cancel_ml_task_run(params = {}) ⇒ Types::CancelMLTaskRunResponse

Cancels (stops) a task run. Machine learning task runs are asynchronous tasks that Glue runs on your behalf as part of various machine learning workflows. You can cancel a machine learning task run at any time by calling CancelMLTaskRun with a task run's parent transform's TransformID and the task run's TaskRunId.

Examples:

Request syntax with placeholder values

resp = client.cancel_ml_task_run({
  transform_id: "HashString", # required
  task_run_id: "HashString", # required
})

Response structure

resp.transform_id #=> String
resp.task_run_id #=> String
resp.status #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED", "TIMEOUT"

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :transform_id (required, String) —

  The unique identifier of the machine learning transform.

• :task_run_id (required, String) —

  A unique identifier for the task run.

Returns:

• (Types::CancelMLTaskRunResponse) —

  Returns a response object which responds to the following methods:

  • #transform_id => String
  • #task_run_id => String
  • #status => String

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2000

def cancel_ml_task_run(params = {}, options = {})
  req = build_request(:cancel_ml_task_run, params)
  req.send_request(options)
end

#cancel_statement(params = {}) ⇒ Struct

Cancels the statement..

Examples:

Request syntax with placeholder values

resp = client.cancel_statement({
  session_id: "NameString", # required
  id: 1, # required
  request_origin: "OrchestrationNameString",
})

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :session_id (required, String) —

  The Session ID of the statement to be cancelled.

• :id (required, Integer) —

  The ID of the statement to be cancelled.

• :request_origin (String) —

  The origin of the request to cancel the statement.

Returns:

• (Struct) —

  Returns an empty response.

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2030

def cancel_statement(params = {}, options = {})
  req = build_request(:cancel_statement, params)
  req.send_request(options)
end

#check_schema_version_validity(params = {}) ⇒ Types::CheckSchemaVersionValidityResponse

Validates the supplied schema. This call has no side effects, it simply validates using the supplied schema using DataFormat as the format. Since it does not take a schema set name, no compatibility checks are performed.

Examples:

Request syntax with placeholder values

resp = client.check_schema_version_validity({
  data_format: "AVRO", # required, accepts AVRO, JSON, PROTOBUF
  schema_definition: "SchemaDefinitionString", # required
})

Response structure

resp.valid #=> Boolean
resp.error #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :data_format (required, String) —

  The data format of the schema definition. Currently AVRO, JSON and PROTOBUF are supported.

• :schema_definition (required, String) —

  The definition of the schema that has to be validated.

Returns:

• (Types::CheckSchemaVersionValidityResponse) —

  Returns a response object which responds to the following methods:

  • #valid => Boolean
  • #error => String

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2068

def check_schema_version_validity(params = {}, options = {})
  req = build_request(:check_schema_version_validity, params)
  req.send_request(options)
end

#create_blueprint(params = {}) ⇒ Types::CreateBlueprintResponse

Registers a blueprint with Glue.

Examples:

Request syntax with placeholder values

resp = client.create_blueprint({
  name: "OrchestrationNameString", # required
  description: "Generic512CharString",
  blueprint_location: "OrchestrationS3Location", # required
  tags: {
    "TagKey" => "TagValue",
  },
})

Response structure

resp.name #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :name (required, String) —

  The name of the blueprint.

• :description (String) —

  A description of the blueprint.

• :blueprint_location (required, String) —

  Specifies a path in Amazon S3 where the blueprint is published.

• :tags (Hash<String,String>) —

  The tags to be applied to this blueprint.

Returns:

• (Types::CreateBlueprintResponse) —

  Returns a response object which responds to the following methods:

  • #name => String

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2110

def create_blueprint(params = {}, options = {})
  req = build_request(:create_blueprint, params)
  req.send_request(options)
end

#create_classifier(params = {}) ⇒ Struct

Creates a classifier in the user's account. This can be a GrokClassifier, an XMLClassifier, a JsonClassifier, or a CsvClassifier, depending on which field of the request is present.

Examples:

Request syntax with placeholder values

resp = client.create_classifier({
  grok_classifier: {
    classification: "Classification", # required
    name: "NameString", # required
    grok_pattern: "GrokPattern", # required
    custom_patterns: "CustomPatterns",
  },
  xml_classifier: {
    classification: "Classification", # required
    name: "NameString", # required
    row_tag: "RowTag",
  },
  json_classifier: {
    name: "NameString", # required
    json_path: "JsonPath", # required
  },
  csv_classifier: {
    name: "NameString", # required
    delimiter: "CsvColumnDelimiter",
    quote_symbol: "CsvQuoteSymbol",
    contains_header: "UNKNOWN", # accepts UNKNOWN, PRESENT, ABSENT
    header: ["NameString"],
    disable_value_trimming: false,
    allow_single_column: false,
  },
})

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :grok_classifier (Types::CreateGrokClassifierRequest) —

  A GrokClassifier object specifying the classifier to create.

• :xml_classifier (Types::CreateXMLClassifierRequest) —

  An XMLClassifier object specifying the classifier to create.

• :json_classifier (Types::CreateJsonClassifierRequest) —

  A JsonClassifier object specifying the classifier to create.

• :csv_classifier (Types::CreateCsvClassifierRequest) —

  A CsvClassifier object specifying the classifier to create.

Returns:

• (Struct) —

  Returns an empty response.

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2166

def create_classifier(params = {}, options = {})
  req = build_request(:create_classifier, params)
  req.send_request(options)
end

#create_connection(params = {}) ⇒ Struct

Creates a connection definition in the Data Catalog.

Examples:

Request syntax with placeholder values

resp = client.create_connection({
  catalog_id: "CatalogIdString",
  connection_input: { # required
    name: "NameString", # required
    description: "DescriptionString",
    connection_type: "JDBC", # required, accepts JDBC, SFTP, MONGODB, KAFKA, NETWORK, MARKETPLACE, CUSTOM
    match_criteria: ["NameString"],
    connection_properties: { # required
      "HOST" => "ValueString",
    },
    physical_connection_requirements: {
      subnet_id: "NameString",
      security_group_id_list: ["NameString"],
      availability_zone: "NameString",
    },
  },
  tags: {
    "TagKey" => "TagValue",
  },
})

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog in which to create the connection. If none is provided, the Amazon Web Services account ID is used by default.

• :connection_input (required, Types::ConnectionInput) —

  A ConnectionInput object defining the connection to create.

• :tags (Hash<String,String>) —

  The tags you assign to the connection.

Returns:

• (Struct) —

  Returns an empty response.

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2212

def create_connection(params = {}, options = {})
  req = build_request(:create_connection, params)
  req.send_request(options)
end

#create_crawler(params = {}) ⇒ Struct

Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in the s3Targets field, the jdbcTargets field, or the DynamoDBTargets field.

Examples:

Request syntax with placeholder values

resp = client.create_crawler({
  name: "NameString", # required
  role: "Role", # required
  database_name: "DatabaseName",
  description: "DescriptionString",
  targets: { # required
    s3_targets: [
      {
        path: "Path",
        exclusions: ["Path"],
        connection_name: "ConnectionName",
        sample_size: 1,
        event_queue_arn: "EventQueueArn",
        dlq_event_queue_arn: "EventQueueArn",
      },
    ],
    jdbc_targets: [
      {
        connection_name: "ConnectionName",
        path: "Path",
        exclusions: ["Path"],
      },
    ],
    mongo_db_targets: [
      {
        connection_name: "ConnectionName",
        path: "Path",
        scan_all: false,
      },
    ],
    dynamo_db_targets: [
      {
        path: "Path",
        scan_all: false,
        scan_rate: 1.0,
      },
    ],
    catalog_targets: [
      {
        database_name: "NameString", # required
        tables: ["NameString"], # required
        connection_name: "ConnectionName",
      },
    ],
    delta_targets: [
      {
        delta_tables: ["Path"],
        connection_name: "ConnectionName",
        write_manifest: false,
      },
    ],
  },
  schedule: "CronExpression",
  classifiers: ["NameString"],
  table_prefix: "TablePrefix",
  schema_change_policy: {
    update_behavior: "LOG", # accepts LOG, UPDATE_IN_DATABASE
    delete_behavior: "LOG", # accepts LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
  },
  recrawl_policy: {
    recrawl_behavior: "CRAWL_EVERYTHING", # accepts CRAWL_EVERYTHING, CRAWL_NEW_FOLDERS_ONLY, CRAWL_EVENT_MODE
  },
  lineage_configuration: {
    crawler_lineage_settings: "ENABLE", # accepts ENABLE, DISABLE
  },
  lake_formation_configuration: {
    use_lake_formation_credentials: false,
    account_id: "AccountId",
  },
  configuration: "CrawlerConfiguration",
  crawler_security_configuration: "CrawlerSecurityConfiguration",
  tags: {
    "TagKey" => "TagValue",
  },
})

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :name (required, String) —

  Name of the new crawler.

• :role (required, String) —

  The IAM role or Amazon Resource Name (ARN) of an IAM role used by the new crawler to access customer resources.

• :database_name (String) —

  The Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/*.

• :description (String) —

  A description of the new crawler.

• :targets (required, Types::CrawlerTargets) —

  A list of collection of targets to crawl.

• :schedule (String) —

  A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

• :classifiers (Array<String>) —

  A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

• :table_prefix (String) —

  The table prefix used for catalog tables that are created.

• :schema_change_policy (Types::SchemaChangePolicy) —

  The policy for the crawler's update and deletion behavior.

• :recrawl_policy (Types::RecrawlPolicy) —

  A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

• :lineage_configuration (Types::LineageConfiguration) —

  Specifies data lineage configuration settings for the crawler.

• :lake_formation_configuration (Types::LakeFormationConfiguration) —

  Specifies Lake Formation configuration settings for the crawler.

• :configuration (String) —

  Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

• :crawler_security_configuration (String) —

  The name of the SecurityConfiguration structure to be used by this crawler.

• :tags (Hash<String,String>) —

  The tags to use with this crawler request. You may use tags to limit access to the crawler. For more information about tags in Glue, see Amazon Web Services Tags in Glue in the developer guide.

Returns:

• (Struct) —

  Returns an empty response.

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2376

def create_crawler(params = {}, options = {})
  req = build_request(:create_crawler, params)
  req.send_request(options)
end

#create_custom_entity_type(params = {}) ⇒ Types::CreateCustomEntityTypeResponse

Creates a custom pattern that is used to detect sensitive data across the columns and rows of your structured data.

Each custom pattern you create specifies a regular expression and an optional list of context words. If no context words are passed only a regular expression is checked.

Examples:

Request syntax with placeholder values

resp = client.create_custom_entity_type({
  name: "NameString", # required
  regex_string: "NameString", # required
  context_words: ["NameString"],
})

Response structure

resp.name #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :name (required, String) —

  A name for the custom pattern that allows it to be retrieved or deleted later. This name must be unique per Amazon Web Services account.

• :regex_string (required, String) —

  A regular expression string that is used for detecting sensitive data in a custom pattern.

• :context_words (Array<String>) —

  A list of context words. If none of these context words are found within the vicinity of the regular expression the data will not be detected as sensitive data.

  If no context words are passed only a regular expression is checked.

Returns:

• (Types::CreateCustomEntityTypeResponse) —

  Returns a response object which responds to the following methods:

  • #name => String

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2424

def create_custom_entity_type(params = {}, options = {})
  req = build_request(:create_custom_entity_type, params)
  req.send_request(options)
end

#create_database(params = {}) ⇒ Struct

Creates a new database in a Data Catalog.

Examples:

Request syntax with placeholder values

resp = client.create_database({
  catalog_id: "CatalogIdString",
  database_input: { # required
    name: "NameString", # required
    description: "DescriptionString",
    location_uri: "URI",
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
    create_table_default_permissions: [
      {
        principal: {
          data_lake_principal_identifier: "DataLakePrincipalString",
        },
        permissions: ["ALL"], # accepts ALL, SELECT, ALTER, DROP, DELETE, INSERT, CREATE_DATABASE, CREATE_TABLE, DATA_LOCATION_ACCESS
      },
    ],
    target_database: {
      catalog_id: "CatalogIdString",
      database_name: "NameString",
    },
  },
})

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :catalog_id (String) —

  The ID of the Data Catalog in which to create the database. If none is provided, the Amazon Web Services account ID is used by default.

• :database_input (required, Types::DatabaseInput) —

  The metadata for the database.

Returns:

• (Struct) —

  Returns an empty response.

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2470

def create_database(params = {}, options = {})
  req = build_request(:create_database, params)
  req.send_request(options)
end

#create_dev_endpoint(params = {}) ⇒ Types::CreateDevEndpointResponse

Creates a new development endpoint.

Examples:

Request syntax with placeholder values

resp = client.create_dev_endpoint({
  endpoint_name: "GenericString", # required
  role_arn: "RoleArn", # required
  security_group_ids: ["GenericString"],
  subnet_id: "GenericString",
  public_key: "GenericString",
  public_keys: ["GenericString"],
  number_of_nodes: 1,
  worker_type: "Standard", # accepts Standard, G.1X, G.2X
  glue_version: "GlueVersionString",
  number_of_workers: 1,
  extra_python_libs_s3_path: "GenericString",
  extra_jars_s3_path: "GenericString",
  security_configuration: "NameString",
  tags: {
    "TagKey" => "TagValue",
  },
  arguments: {
    "GenericString" => "GenericString",
  },
})

Response structure

resp.endpoint_name #=> String
resp.status #=> String
resp.security_group_ids #=> Array
resp.security_group_ids[0] #=> String
resp.subnet_id #=> String
resp.role_arn #=> String
resp.yarn_endpoint_address #=> String
resp.zeppelin_remote_spark_interpreter_port #=> Integer
resp.number_of_nodes #=> Integer
resp.worker_type #=> String, one of "Standard", "G.1X", "G.2X"
resp.glue_version #=> String
resp.number_of_workers #=> Integer
resp.availability_zone #=> String
resp.vpc_id #=> String
resp.extra_python_libs_s3_path #=> String
resp.extra_jars_s3_path #=> String
resp.failure_reason #=> String
resp.security_configuration #=> String
resp.created_timestamp #=> Time
resp.arguments #=> Hash
resp.arguments["GenericString"] #=> String

Parameters:

• params (Hash) (defaults to: {}) —

  ({})

Options Hash (params):

• :endpoint_name (required, String) —

  The name to be assigned to the new DevEndpoint.

• :role_arn (required, String) —

  The IAM role for the DevEndpoint.

• :security_group_ids (Array<String>) —

  Security group IDs for the security groups to be used by the new DevEndpoint.

• :subnet_id (String) —

  The subnet ID for the new DevEndpoint to use.

• :public_key (String) —

  The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility because the recommended attribute to use is public keys.

• :public_keys (Array<String>) —

  A list of public keys to be used by the development endpoints for authentication. The use of this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

  If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the UpdateDevEndpoint API with the public key content in the deletePublicKeys attribute, and the list of new keys in the addPublicKeys attribute.

• :number_of_nodes (Integer) —

  The number of Glue Data Processing Units (DPUs) to allocate to this DevEndpoint.

• :worker_type (String) —

  The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

  • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

  • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

  • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

  Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

• :glue_version (String) —

  Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints.

  For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

  Development endpoints that are created without specifying a Glue version default to Glue 0.9.

  You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

• :number_of_workers (Integer) —

  The number of workers of a defined workerType that are allocated to the development endpoint.

  The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

• :extra_python_libs_s3_path (String) —

  The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

  You can only use pure Python libraries with a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

• :extra_jars_s3_path (String) —

  The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint.

• :security_configuration (String) —

  The name of the SecurityConfiguration structure to be used with this DevEndpoint.

• :tags (Hash<String,String>) —

  The tags to use with this DevEndpoint. You may use tags to limit access to the DevEndpoint. For more information about tags in Glue, see Amazon Web Services Tags in Glue in the developer guide.

• :arguments (Hash<String,String>) —

  A map of arguments used to configure the DevEndpoint.

Returns:

• (Types::CreateDevEndpointResponse) —

  Returns a response object which responds to the following methods:

  • #endpoint_name => String
  • #status => String
  • #security_group_ids => Array<String>
  • #subnet_id => String
  • #role_arn => String
  • #yarn_endpoint_address => String
  • #zeppelin_remote_spark_interpreter_port => Integer
  • #number_of_nodes => Integer
  • #worker_type => String
  • #glue_version => String
  • #number_of_workers => Integer
  • #availability_zone => String
  • #vpc_id => String
  • #extra_python_libs_s3_path => String
  • #extra_jars_s3_path => String
  • #failure_reason => String
  • #security_configuration => String
  • #created_timestamp => Time
  • #arguments => Hash<String,String>

See Also:

• AWS API Documentation

# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2669

def create_dev_endpoint(params = {}, options = {})
  req = build_request(:create_dev_endpoint, params)
  req.send_request(options)
end

#create_job(params = {}) ⇒ Types::CreateJobResponse

Creates a new job definition.

Examples:

Request syntax with placeholder values

resp = client.create_job({
  name: "NameString", # required
  description: "DescriptionString",
  log_uri: "UriString",
  role: "RoleString", # required
  execution_property: {
    max_concurrent_runs: 1,
  },
  command: { # required
    name: "GenericString",
    script_location: "ScriptLocationString",
    python_version: "PythonVersionString",
  },
  default_arguments: {
    "GenericString" => "GenericString",
  },
  non_overridable_arguments: {
    "GenericString" => "GenericString",
  },
  connections: {
    connections: ["GenericString"],
  },
  max_retries: 1,
  allocated_capacity: 1,
  timeout: 1,
  max_capacity: 1.0,
  security_configuration: "NameString",
  tags: {
    "TagKey" => "TagValue",
  },
  notification_property: {
    notify_delay_after: 1,
  },
  glue_version: "GlueVersionString",
  number_of_workers: 1,
  worker_type: "Standard", # accepts Standard, G.1X, G.2X
  code_gen_configuration_nodes: {
    "NodeId" => {
      athena_connector_source: {
        name: "NodeName", # required
        connection_name: "EnclosedInStringProperty", # required
        connector_name: "EnclosedInStringProperty", # required
        connection_type: "EnclosedInStringProperty", # required
        connection_table: "EnclosedInStringPropertyWithQuote",
        schema_name: "EnclosedInStringProperty", # required
        output_schemas: [
          {
            columns: [
              {
                name: "GlueStudioColumnNameString", # required
                type: "ColumnTypeString",
              },
            ],
          },
        ],
      },
      jdbc_connector_source: {
        name: "NodeName", # required
        connection_name: "EnclosedInStringProperty", # required
        connector_name: "EnclosedInStringProperty", # required
        connection_type: "EnclosedInStringProperty", # required
        additional_options: {
          filter_predicate: "EnclosedInStringProperty",
          partition_column: "EnclosedInStringProperty",
          lower_bound: 1,
          upper_bound: 1,
          num_partitions: 1,
          job_bookmark_keys: ["EnclosedInStringProperty"],
          job_bookmark_keys_sort_order: "EnclosedInStringProperty",
          data_type_mapping: {
            "ARRAY" => "DATE", # accepts DATE, STRING, TIMESTAMP, INT, FLOAT, LONG, BIGDECIMAL, BYTE, SHORT, DOUBLE
          },
        },
        connection_table: "EnclosedInStringPropertyWithQuote",
        query: "SqlQuery",
        output_schemas: [
          {
            columns: [
              {
                name: "GlueStudioColumnNameString", # required
                type: "ColumnTypeString",
              },
            ],
          },
        ],
      },
      spark_connector_source: {
        name: "NodeName", # required
        connection_name: "EnclosedInStringProperty", # required
        connector_name: "EnclosedInStringProperty", # required
        connection_type: "EnclosedInStringProperty", # required
        additional_options: {
          "EnclosedInStringProperty" => "EnclosedInStringProperty",
        },
        output_schemas: [
          {
            columns: [
              {
                name: "GlueStudioColumnNameString", # required
                type: "ColumnTypeString",
              },
            ],
          },
        ],
      },
      catalog_source: {
        name: "NodeName", # required
        database: "EnclosedInStringProperty", # required
        table: "EnclosedInStringProperty", # required
      },
      redshift_source: {
        name: "NodeName", # required
        database: "EnclosedInStringProperty", # required
        table: "EnclosedInStringProperty", # required
        redshift_tmp_dir: "EnclosedInStringProperty",
        tmp_dir_iam_role: "EnclosedInStringProperty",
      },
      s3_catalog_source: {
        name: "NodeName", # required
        database: "EnclosedInStringProperty", # required
        table: "EnclosedInStringProperty", # required
        partition_predicate: "EnclosedInStringProperty",
        additional_options: {
          bounded_size: 1,
          bounded_files: 1,
        },
      },
      s3_csv_source: {
        name: "NodeName", # required
        paths: ["EnclosedInStringProperty"], # required
        compression_type: "gzip", # accepts gzip, bzip2
        exclusions: ["EnclosedInStringProperty"],
        group_size: "EnclosedInStringProperty",
        group_files: "EnclosedInStringProperty",
        recurse: false,
        max_band: 1,
        max_files_in_band: 1,
        additional_options: {
          bounded_size: 1,
          bounded_files: 1,
          enable_sample_path: false,
          sample_path: "EnclosedInStringProperty",
        },
        separator: "comma", # required, accepts comma, ctrla, pipe, semicolon, tab
        escaper: "EnclosedInStringPropertyWithQuote",
        quote_char: "quote", # required, accepts quote, quillemet, single_quote, disabled
        multiline: false,
        with_header: false,
        write_header: false,
        skip_first: false,
        optimize_performance: false,
        output_schemas: [
          {
            columns: [
              {
                name: "GlueStudioColumnNameString", # required
                type: "ColumnTypeString",
              },
            ],
          },
        ],
      },
      s3_json_source: {
        name: "NodeName", # required
        paths: ["EnclosedInStringProperty"], # required
        compression_type: "gzip", # accepts gzip, bzip2
        exclusions: ["EnclosedInStringProperty"],
        group_size: "EnclosedInStringProperty",
        group_files: "EnclosedInStringProperty",
        recurse: false,
        max_band: 1,
        max_files_in_band: 1,
        additional_options: {
          bounded_size: 1,
          bounded_files: 1,
          enable_sample_path: false,
          sample_path: "EnclosedInStringProperty",
        },
        json_path: "EnclosedInStringProperty",
        multiline: false,
        output_schemas: [
          {
            columns: [
              {
                name: "GlueStudioColumnNameString", # required
                type: "ColumnTypeString",
              },
            ],
          },
        ],
      },
      s3_parquet_source: {
        name: "NodeName", # required
        paths: ["EnclosedInStringProperty"], # required
        compression_type: "snappy", # accepts snappy, lzo, gzip, uncompressed, none
        exclusions: ["EnclosedInStringProperty"],
        group_size: "EnclosedInStringProperty",
        group_files: "EnclosedInStringProperty",
        recurse: false,
        max_band: 1,
        max_files_in_band: 1,
        additional_options: {
          bounded_size: 1