You are viewing documentation for version 3 of the AWS SDK for Ruby. Version 2 documentation can be found here.

Class: Aws::Glue::Client

Inherits:
Seahorse::Client::Base show all
Includes:
ClientStubs
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb

Instance Attribute Summary

Attributes inherited from Seahorse::Client::Base

#config, #handlers

API Operations collapse

Instance Method Summary collapse

Methods included from ClientStubs

#stub_data, #stub_responses

Methods inherited from Seahorse::Client::Base

add_plugin, api, clear_plugins, define, new, #operation_names, plugins, remove_plugin, set_api, set_plugins

Methods included from Seahorse::Client::HandlerBuilder

#handle, #handle_request, #handle_response

Constructor Details

#initialize(*args) ⇒ Client

Returns a new instance of Client

Parameters:

  • options (Hash)

    a customizable set of options



152
153
154
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 152

def initialize(*args)
  super
end

Instance Method Details

#batch_create_partition(params = {}) ⇒ Types::BatchCreatePartitionResponse

Creates one or more partitions in a batch operation.

Examples:

Request syntax with placeholder values


resp = client.batch_create_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partition_input_list: [ # required
    {
      values: ["ValueString"],
      last_access_time: Time.now,
      storage_descriptor: {
        columns: [
          {
            name: "NameString", # required
            type: "ColumnTypeString",
            comment: "CommentString",
          },
        ],
        location: "LocationString",
        input_format: "FormatString",
        output_format: "FormatString",
        compressed: false,
        number_of_buckets: 1,
        serde_info: {
          name: "NameString",
          serialization_library: "NameString",
          parameters: {
            "KeyString" => "ParametersMapValue",
          },
        },
        bucket_columns: ["NameString"],
        sort_columns: [
          {
            column: "NameString", # required
            sort_order: 1, # required
          },
        ],
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
        skewed_info: {
          skewed_column_names: ["NameString"],
          skewed_column_values: ["ColumnValuesString"],
          skewed_column_value_location_maps: {
            "ColumnValuesString" => "ColumnValuesString",
          },
        },
        stored_as_sub_directories: false,
      },
      parameters: {
        "KeyString" => "ParametersMapValue",
      },
      last_analyzed_time: Time.now,
    },
  ],
})

Response structure


resp.errors #=> Array
resp.errors[0].partition_values #=> Array
resp.errors[0].partition_values[0] #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the catalog in which the partion is to be created. Currently, this should be the AWS account ID.

  • :database_name (required, String)

    The name of the metadata database in which the partition is to be created.

  • :table_name (required, String)

    The name of the metadata table in which the partition is to be created.

  • :partition_input_list (required, Array<Types::PartitionInput>)

    A list of PartitionInput structures that define the partitions to be created.

Returns:

See Also:



249
250
251
252
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 249

def batch_create_partition(params = {}, options = {})
  req = build_request(:batch_create_partition, params)
  req.send_request(options)
end

#batch_delete_connection(params = {}) ⇒ Types::BatchDeleteConnectionResponse

Deletes a list of connection definitions from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.batch_delete_connection({
  catalog_id: "CatalogIdString",
  connection_name_list: ["NameString"], # required
})

Response structure


resp.succeeded #=> Array
resp.succeeded[0] #=> String
resp.errors #=> Hash
resp.errors["NameString"].error_code #=> String
resp.errors["NameString"].error_message #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the connections reside. If none is supplied, the AWS account ID is used by default.

  • :connection_name_list (required, Array<String>)

    A list of names of the connections to delete.

Returns:

See Also:



287
288
289
290
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 287

def batch_delete_connection(params = {}, options = {})
  req = build_request(:batch_delete_connection, params)
  req.send_request(options)
end

#batch_delete_partition(params = {}) ⇒ Types::BatchDeletePartitionResponse

Deletes one or more partitions in a batch operation.

Examples:

Request syntax with placeholder values


resp = client.batch_delete_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partitions_to_delete: [ # required
    {
      values: ["ValueString"], # required
    },
  ],
})

Response structure


resp.errors #=> Array
resp.errors[0].partition_values #=> Array
resp.errors[0].partition_values[0] #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the partition to be deleted resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database in which the table in question resides.

  • :table_name (required, String)

    The name of the table where the partitions to be deleted is located.

  • :partitions_to_delete (required, Array<Types::PartitionValueList>)

    A list of PartitionInput structures that define the partitions to be deleted.

Returns:

See Also:



338
339
340
341
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 338

def batch_delete_partition(params = {}, options = {})
  req = build_request(:batch_delete_partition, params)
  req.send_request(options)
end

#batch_delete_table(params = {}) ⇒ Types::BatchDeleteTableResponse

Deletes multiple tables at once.

Examples:

Request syntax with placeholder values


resp = client.batch_delete_table({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  tables_to_delete: ["NameString"], # required
})

Response structure


resp.errors #=> Array
resp.errors[0].table_name #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the tables to delete reside.

  • :tables_to_delete (required, Array<String>)

    A list of the table to delete.

Returns:

See Also:



378
379
380
381
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 378

def batch_delete_table(params = {}, options = {})
  req = build_request(:batch_delete_table, params)
  req.send_request(options)
end

#batch_get_partition(params = {}) ⇒ Types::BatchGetPartitionResponse

Retrieves partitions in a batch request.

Examples:

Request syntax with placeholder values


resp = client.batch_get_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partitions_to_get: [ # required
    {
      values: ["ValueString"], # required
    },
  ],
})

Response structure


resp.partitions #=> Array
resp.partitions[0].values #=> Array
resp.partitions[0].values[0] #=> String
resp.partitions[0].database_name #=> String
resp.partitions[0].table_name #=> String
resp.partitions[0].creation_time #=> Time
resp.partitions[0].last_access_time #=> Time
resp.partitions[0].storage_descriptor.columns #=> Array
resp.partitions[0].storage_descriptor.columns[0].name #=> String
resp.partitions[0].storage_descriptor.columns[0].type #=> String
resp.partitions[0].storage_descriptor.columns[0].comment #=> String
resp.partitions[0].storage_descriptor.location #=> String
resp.partitions[0].storage_descriptor.input_format #=> String
resp.partitions[0].storage_descriptor.output_format #=> String
resp.partitions[0].storage_descriptor.compressed #=> Boolean
resp.partitions[0].storage_descriptor.number_of_buckets #=> Integer
resp.partitions[0].storage_descriptor.serde_info.name #=> String
resp.partitions[0].storage_descriptor.serde_info.serialization_library #=> String
resp.partitions[0].storage_descriptor.serde_info.parameters #=> Hash
resp.partitions[0].storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.bucket_columns #=> Array
resp.partitions[0].storage_descriptor.bucket_columns[0] #=> String
resp.partitions[0].storage_descriptor.sort_columns #=> Array
resp.partitions[0].storage_descriptor.sort_columns[0].column #=> String
resp.partitions[0].storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.partitions[0].storage_descriptor.parameters #=> Hash
resp.partitions[0].storage_descriptor.parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.partitions[0].storage_descriptor.stored_as_sub_directories #=> Boolean
resp.partitions[0].parameters #=> Hash
resp.partitions[0].parameters["KeyString"] #=> String
resp.partitions[0].last_analyzed_time #=> Time
resp.unprocessed_keys #=> Array
resp.unprocessed_keys[0].values #=> Array
resp.unprocessed_keys[0].values[0] #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the partitions reside.

  • :table_name (required, String)

    The name of the partitions' table.

  • :partitions_to_get (required, Array<Types::PartitionValueList>)

    A list of partition values identifying the partitions to retrieve.

Returns:

See Also:



463
464
465
466
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 463

def batch_get_partition(params = {}, options = {})
  req = build_request(:batch_get_partition, params)
  req.send_request(options)
end

#batch_stop_job_run(params = {}) ⇒ Types::BatchStopJobRunResponse

Stops a batch of job runs for a given job.

Examples:

Request syntax with placeholder values


resp = client.batch_stop_job_run({
  job_name: "NameString", # required
  job_run_ids: ["IdString"], # required
})

Response structure


resp.successful_submissions #=> Array
resp.successful_submissions[0].job_name #=> String
resp.successful_submissions[0].job_run_id #=> String
resp.errors #=> Array
resp.errors[0].job_name #=> String
resp.errors[0].job_run_id #=> String
resp.errors[0].error_detail.error_code #=> String
resp.errors[0].error_detail.error_message #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    The name of the job whose job runs are to be stopped.

  • :job_run_ids (required, Array<String>)

    A list of job run Ids of the given job to be stopped.

Returns:

See Also:



503
504
505
506
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 503

def batch_stop_job_run(params = {}, options = {})
  req = build_request(:batch_stop_job_run, params)
  req.send_request(options)
end

#create_classifier(params = {}) ⇒ Struct

Creates a classifier in the user's account. This may be either a GrokClassifier or an XMLClassifier.

Examples:

Request syntax with placeholder values


resp = client.create_classifier({
  grok_classifier: {
    classification: "Classification", # required
    name: "NameString", # required
    grok_pattern: "GrokPattern", # required
    custom_patterns: "CustomPatterns",
  },
  xml_classifier: {
    classification: "Classification", # required
    name: "NameString", # required
    row_tag: "RowTag",
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

Returns:

  • (Struct)

    Returns an empty response.

See Also:



539
540
541
542
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 539

def create_classifier(params = {}, options = {})
  req = build_request(:create_classifier, params)
  req.send_request(options)
end

#create_connection(params = {}) ⇒ Struct

Creates a connection definition in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.create_connection({
  catalog_id: "CatalogIdString",
  connection_input: { # required
    name: "NameString",
    description: "DescriptionString",
    connection_type: "JDBC", # accepts JDBC, SFTP
    match_criteria: ["NameString"],
    connection_properties: {
      "HOST" => "ValueString",
    },
    physical_connection_requirements: {
      subnet_id: "NameString",
      security_group_id_list: ["NameString"],
      availability_zone: "NameString",
    },
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which to create the connection. If none is supplied, the AWS account ID is used by default.

  • :connection_input (required, Types::ConnectionInput)

    A ConnectionInput object defining the connection to create.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



579
580
581
582
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 579

def create_connection(params = {}, options = {})
  req = build_request(:create_connection, params)
  req.send_request(options)
end

#create_crawler(params = {}) ⇒ Struct

Creates a new crawler with specified targets, role, configuration, and optional schedule. At least one crawl target must be specified, in either the s3Targets or the jdbcTargets field.

Examples:

Request syntax with placeholder values


resp = client.create_crawler({
  name: "NameString", # required
  role: "Role", # required
  database_name: "DatabaseName", # required
  description: "DescriptionString",
  targets: { # required
    s3_targets: [
      {
        path: "Path",
        exclusions: ["Path"],
      },
    ],
    jdbc_targets: [
      {
        connection_name: "ConnectionName",
        path: "Path",
        exclusions: ["Path"],
      },
    ],
  },
  schedule: "CronExpression",
  classifiers: ["NameString"],
  table_prefix: "TablePrefix",
  schema_change_policy: {
    update_behavior: "LOG", # accepts LOG, UPDATE_IN_DATABASE
    delete_behavior: "LOG", # accepts LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
  },
  configuration: "CrawlerConfiguration",
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the new crawler.

  • :role (required, String)

    The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

  • :database_name (required, String)

    The AWS Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/*.

  • :description (String)

    A description of the new crawler.

  • :targets (required, Types::CrawlerTargets)

    A list of collection of targets to crawl.

  • :schedule (String)

    A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

  • :classifiers (Array<String>)

    A list of custom classifiers that the user has registered. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

  • :table_prefix (String)

    The table prefix used for catalog tables that are created.

  • :schema_change_policy (Types::SchemaChangePolicy)

    Policy for the crawler's update and deletion behavior.

  • :configuration (String)

    Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

    You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



673
674
675
676
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 673

def create_crawler(params = {}, options = {})
  req = build_request(:create_crawler, params)
  req.send_request(options)
end

#create_database(params = {}) ⇒ Struct

Creates a new database in a Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.create_database({
  catalog_id: "CatalogIdString",
  database_input: { # required
    name: "NameString", # required
    description: "DescriptionString",
    location_uri: "URI",
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which to create the database. If none is supplied, the AWS account ID is used by default.

  • :database_input (required, Types::DatabaseInput)

    A DatabaseInput object defining the metadata database to create in the catalog.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



708
709
710
711
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 708

def create_database(params = {}, options = {})
  req = build_request(:create_database, params)
  req.send_request(options)
end

#create_dev_endpoint(params = {}) ⇒ Types::CreateDevEndpointResponse

Creates a new DevEndpoint.

Examples:

Request syntax with placeholder values


resp = client.create_dev_endpoint({
  endpoint_name: "GenericString", # required
  role_arn: "RoleArn", # required
  security_group_ids: ["GenericString"],
  subnet_id: "GenericString",
  public_key: "GenericString", # required
  number_of_nodes: 1,
  extra_python_libs_s3_path: "GenericString",
  extra_jars_s3_path: "GenericString",
})

Response structure


resp.endpoint_name #=> String
resp.status #=> String
resp.security_group_ids #=> Array
resp.security_group_ids[0] #=> String
resp.subnet_id #=> String
resp.role_arn #=> String
resp.yarn_endpoint_address #=> String
resp.zeppelin_remote_spark_interpreter_port #=> Integer
resp.number_of_nodes #=> Integer
resp.availability_zone #=> String
resp.vpc_id #=> String
resp.extra_python_libs_s3_path #=> String
resp.extra_jars_s3_path #=> String
resp.failure_reason #=> String
resp.created_timestamp #=> Time

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :endpoint_name (required, String)

    The name to be assigned to the new DevEndpoint.

  • :role_arn (required, String)

    The IAM role for the DevEndpoint.

  • :security_group_ids (Array<String>)

    Security group IDs for the security groups to be used by the new DevEndpoint.

  • :subnet_id (String)

    The subnet ID for the new DevEndpoint to use.

  • :public_key (required, String)

    The public key to use for authentication.

  • :number_of_nodes (Integer)

    The number of AWS Glue Data Processing Units (DPUs) to allocate to this DevEndpoint.

  • :extra_python_libs_s3_path (String)

    Path(s) to one or more Python libraries in an S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

    Please note that only pure Python libraries can currently be used on a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not yet supported.

  • :extra_jars_s3_path (String)

    Path to one or more Java Jars in an S3 bucket that should be loaded in your DevEndpoint.

Returns:

See Also:



804
805
806
807
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 804

def create_dev_endpoint(params = {}, options = {})
  req = build_request(:create_dev_endpoint, params)
  req.send_request(options)
end

#create_job(params = {}) ⇒ Types::CreateJobResponse

Creates a new job.

Examples:

Request syntax with placeholder values


resp = client.create_job({
  name: "NameString", # required
  description: "DescriptionString",
  log_uri: "UriString",
  role: "RoleString", # required
  execution_property: {
    max_concurrent_runs: 1,
  },
  command: { # required
    name: "GenericString",
    script_location: "ScriptLocationString",
  },
  default_arguments: {
    "GenericString" => "GenericString",
  },
  connections: {
    connections: ["GenericString"],
  },
  max_retries: 1,
  allocated_capacity: 1,
})

Response structure


resp.name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name you assign to this job.

  • :description (String)

    Description of the job.

  • :log_uri (String)

    This field is reserved for future use.

  • :role (required, String)

    The role associated with this job.

  • :execution_property (Types::ExecutionProperty)

    An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job.

  • :command (required, Types::JobCommand)

    The JobCommand that executes this job.

  • :default_arguments (Hash<String,String>)

    The default parameters for this job.

  • :connections (Types::ConnectionsList)

    The connections used for this job.

  • :max_retries (Integer)

    The maximum number of times to retry this job if it fails.

  • :allocated_capacity (Integer)

    The number of capacity units allocated to this job.

Returns:

See Also:



878
879
880
881
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 878

def create_job(params = {}, options = {})
  req = build_request(:create_job, params)
  req.send_request(options)
end

#create_partition(params = {}) ⇒ Struct

Creates a new partition.

Examples:

Request syntax with placeholder values


resp = client.create_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partition_input: { # required
    values: ["ValueString"],
    last_access_time: Time.now,
    storage_descriptor: {
      columns: [
        {
          name: "NameString", # required
          type: "ColumnTypeString",
          comment: "CommentString",
        },
      ],
      location: "LocationString",
      input_format: "FormatString",
      output_format: "FormatString",
      compressed: false,
      number_of_buckets: 1,
      serde_info: {
        name: "NameString",
        serialization_library: "NameString",
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
      },
      bucket_columns: ["NameString"],
      sort_columns: [
        {
          column: "NameString", # required
          sort_order: 1, # required
        },
      ],
      parameters: {
        "KeyString" => "ParametersMapValue",
      },
      skewed_info: {
        skewed_column_names: ["NameString"],
        skewed_column_values: ["ColumnValuesString"],
        skewed_column_value_location_maps: {
          "ColumnValuesString" => "ColumnValuesString",
        },
      },
      stored_as_sub_directories: false,
    },
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
    last_analyzed_time: Time.now,
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the catalog in which the partion is to be created. Currently, this should be the AWS account ID.

  • :database_name (required, String)

    The name of the metadata database in which the partition is to be created.

  • :table_name (required, String)

    The name of the metadata table in which the partition is to be created.

  • :partition_input (required, Types::PartitionInput)

    A PartitionInput structure defining the partition to be created.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



961
962
963
964
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 961

def create_partition(params = {}, options = {})
  req = build_request(:create_partition, params)
  req.send_request(options)
end

#create_script(params = {}) ⇒ Types::CreateScriptResponse

Transforms a directed acyclic graph (DAG) into a Python script.

Examples:

Request syntax with placeholder values


resp = client.create_script({
  dag_nodes: [
    {
      id: "CodeGenIdentifier", # required
      node_type: "CodeGenNodeType", # required
      args: [ # required
        {
          name: "CodeGenArgName", # required
          value: "CodeGenArgValue", # required
          param: false,
        },
      ],
      line_number: 1,
    },
  ],
  dag_edges: [
    {
      source: "CodeGenIdentifier", # required
      target: "CodeGenIdentifier", # required
      target_parameter: "CodeGenArgName",
    },
  ],
})

Response structure


resp.python_script #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

Returns:

See Also:



1012
1013
1014
1015
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1012

def create_script(params = {}, options = {})
  req = build_request(:create_script, params)
  req.send_request(options)
end

#create_table(params = {}) ⇒ Struct

Creates a new table definition in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.create_table({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_input: { # required
    name: "NameString", # required
    description: "DescriptionString",
    owner: "NameString",
    last_access_time: Time.now,
    last_analyzed_time: Time.now,
    retention: 1,
    storage_descriptor: {
      columns: [
        {
          name: "NameString", # required
          type: "ColumnTypeString",
          comment: "CommentString",
        },
      ],
      location: "LocationString",
      input_format: "FormatString",
      output_format: "FormatString",
      compressed: false,
      number_of_buckets: 1,
      serde_info: {
        name: "NameString",
        serialization_library: "NameString",
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
      },
      bucket_columns: ["NameString"],
      sort_columns: [
        {
          column: "NameString", # required
          sort_order: 1, # required
        },
      ],
      parameters: {
        "KeyString" => "ParametersMapValue",
      },
      skewed_info: {
        skewed_column_names: ["NameString"],
        skewed_column_values: ["ColumnValuesString"],
        skewed_column_value_location_maps: {
          "ColumnValuesString" => "ColumnValuesString",
        },
      },
      stored_as_sub_directories: false,
    },
    partition_keys: [
      {
        name: "NameString", # required
        type: "ColumnTypeString",
        comment: "CommentString",
      },
    ],
    view_original_text: "ViewTextString",
    view_expanded_text: "ViewTextString",
    table_type: "TableTypeString",
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which to create the Table. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The catalog database in which to create the new table.

  • :table_input (required, Types::TableInput)

    The TableInput object that defines the metadata table to create in the catalog.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1103
1104
1105
1106
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1103

def create_table(params = {}, options = {})
  req = build_request(:create_table, params)
  req.send_request(options)
end

#create_trigger(params = {}) ⇒ Types::CreateTriggerResponse

Creates a new trigger.

Examples:

Request syntax with placeholder values


resp = client.create_trigger({
  name: "NameString", # required
  type: "SCHEDULED", # required, accepts SCHEDULED, CONDITIONAL, ON_DEMAND
  schedule: "GenericString",
  predicate: {
    logical: "AND", # accepts AND
    conditions: [
      {
        logical_operator: "EQUALS", # accepts EQUALS
        job_name: "NameString",
        state: "STARTING", # accepts STARTING, RUNNING, STOPPING, STOPPED, SUCCEEDED, FAILED
      },
    ],
  },
  actions: [ # required
    {
      job_name: "NameString",
      arguments: {
        "GenericString" => "GenericString",
      },
    },
  ],
  description: "DescriptionString",
})

Response structure


resp.name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name to assign to the new trigger.

  • :type (required, String)

    The type of the new trigger.

  • :schedule (String)

    A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

  • :predicate (Types::Predicate)

    A predicate to specify when the new trigger should fire.

  • :actions (required, Array<Types::Action>)

    The actions initiated by this trigger when it fires.

  • :description (String)

    A description of the new trigger.

Returns:

See Also:



1173
1174
1175
1176
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1173

def create_trigger(params = {}, options = {})
  req = build_request(:create_trigger, params)
  req.send_request(options)
end

#create_user_defined_function(params = {}) ⇒ Struct

Creates a new function definition in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.create_user_defined_function({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  function_input: { # required
    function_name: "NameString",
    class_name: "NameString",
    owner_name: "NameString",
    owner_type: "USER", # accepts USER, ROLE, GROUP
    resource_uris: [
      {
        resource_type: "JAR", # accepts JAR, FILE, ARCHIVE
        uri: "URI",
      },
    ],
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which to create the function. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database in which to create the function.

  • :function_input (required, Types::UserDefinedFunctionInput)

    A FunctionInput object that defines the function to create in the Data Catalog.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1216
1217
1218
1219
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1216

def create_user_defined_function(params = {}, options = {})
  req = build_request(:create_user_defined_function, params)
  req.send_request(options)
end

#delete_classifier(params = {}) ⇒ Struct

Removes a classifier from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.delete_classifier({
  name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the classifier to remove.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1238
1239
1240
1241
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1238

def delete_classifier(params = {}, options = {})
  req = build_request(:delete_classifier, params)
  req.send_request(options)
end

#delete_connection(params = {}) ⇒ Struct

Deletes a connection from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.delete_connection({
  catalog_id: "CatalogIdString",
  connection_name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the connection resides. If none is supplied, the AWS account ID is used by default.

  • :connection_name (required, String)

    The name of the connection to delete.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1265
1266
1267
1268
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1265

def delete_connection(params = {}, options = {})
  req = build_request(:delete_connection, params)
  req.send_request(options)
end

#delete_crawler(params = {}) ⇒ Struct

Removes a specified crawler from the Data Catalog, unless the crawler state is RUNNING.

Examples:

Request syntax with placeholder values


resp = client.delete_crawler({
  name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the crawler to remove.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1288
1289
1290
1291
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1288

def delete_crawler(params = {}, options = {})
  req = build_request(:delete_crawler, params)
  req.send_request(options)
end

#delete_database(params = {}) ⇒ Struct

Removes a specified Database from a Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.delete_database({
  catalog_id: "CatalogIdString",
  name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the database resides. If none is supplied, the AWS account ID is used by default.

  • :name (required, String)

    The name of the Database to delete.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1315
1316
1317
1318
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1315

def delete_database(params = {}, options = {})
  req = build_request(:delete_database, params)
  req.send_request(options)
end

#delete_dev_endpoint(params = {}) ⇒ Struct

Deletes a specified DevEndpoint.

Examples:

Request syntax with placeholder values


resp = client.delete_dev_endpoint({
  endpoint_name: "GenericString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :endpoint_name (required, String)

    The name of the DevEndpoint.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1337
1338
1339
1340
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1337

def delete_dev_endpoint(params = {}, options = {})
  req = build_request(:delete_dev_endpoint, params)
  req.send_request(options)
end

#delete_job(params = {}) ⇒ Types::DeleteJobResponse

Deletes a specified job.

Examples:

Request syntax with placeholder values


resp = client.delete_job({
  job_name: "NameString", # required
})

Response structure


resp.job_name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    The name of the job to delete.

Returns:

See Also:



1365
1366
1367
1368
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1365

def delete_job(params = {}, options = {})
  req = build_request(:delete_job, params)
  req.send_request(options)
end

#delete_partition(params = {}) ⇒ Struct

Deletes a specified partition.

Examples:

Request syntax with placeholder values


resp = client.delete_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partition_values: ["ValueString"], # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the partition to be deleted resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database in which the table in question resides.

  • :table_name (required, String)

    The name of the table where the partition to be deleted is located.

  • :partition_values (required, Array<String>)

    The values that define the partition.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1401
1402
1403
1404
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1401

def delete_partition(params = {}, options = {})
  req = build_request(:delete_partition, params)
  req.send_request(options)
end

#delete_table(params = {}) ⇒ Struct

Removes a table definition from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.delete_table({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database in which the table resides.

  • :name (required, String)

    The name of the table to be deleted.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1432
1433
1434
1435
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1432

def delete_table(params = {}, options = {})
  req = build_request(:delete_table, params)
  req.send_request(options)
end

#delete_trigger(params = {}) ⇒ Types::DeleteTriggerResponse

Deletes a specified trigger.

Examples:

Request syntax with placeholder values


resp = client.delete_trigger({
  name: "NameString", # required
})

Response structure


resp.name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name of the trigger to delete.

Returns:

See Also:



1460
1461
1462
1463
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1460

def delete_trigger(params = {}, options = {})
  req = build_request(:delete_trigger, params)
  req.send_request(options)
end

#delete_user_defined_function(params = {}) ⇒ Struct

Deletes an existing function definition from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.delete_user_defined_function({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  function_name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the function to be deleted is located. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the function is located.

  • :function_name (required, String)

    The name of the function definition to be deleted.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



1491
1492
1493
1494
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1491

def delete_user_defined_function(params = {}, options = {})
  req = build_request(:delete_user_defined_function, params)
  req.send_request(options)
end

#get_catalog_import_status(params = {}) ⇒ Types::GetCatalogImportStatusResponse

Retrieves the status of a migration operation.

Examples:

Request syntax with placeholder values


resp = client.get_catalog_import_status({
  catalog_id: "CatalogIdString",
})

Response structure


resp.import_status.import_completed #=> Boolean
resp.import_status.import_time #=> Time
resp.import_status.imported_by #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the catalog to migrate. Currently, this should be the AWS account ID.

Returns:

See Also:



1522
1523
1524
1525
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1522

def get_catalog_import_status(params = {}, options = {})
  req = build_request(:get_catalog_import_status, params)
  req.send_request(options)
end

#get_classifier(params = {}) ⇒ Types::GetClassifierResponse

Retrieve a classifier by name.

Examples:

Request syntax with placeholder values


resp = client.get_classifier({
  name: "NameString", # required
})

Response structure


resp.classifier.grok_classifier.name #=> String
resp.classifier.grok_classifier.classification #=> String
resp.classifier.grok_classifier.creation_time #=> Time
resp.classifier.grok_classifier.last_updated #=> Time
resp.classifier.grok_classifier.version #=> Integer
resp.classifier.grok_classifier.grok_pattern #=> String
resp.classifier.grok_classifier.custom_patterns #=> String
resp.classifier.xml_classifier.name #=> String
resp.classifier.xml_classifier.classification #=> String
resp.classifier.xml_classifier.creation_time #=> Time
resp.classifier.xml_classifier.last_updated #=> Time
resp.classifier.xml_classifier.version #=> Integer
resp.classifier.xml_classifier.row_tag #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the classifier to retrieve.

Returns:

See Also:



1562
1563
1564
1565
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1562

def get_classifier(params = {}, options = {})
  req = build_request(:get_classifier, params)
  req.send_request(options)
end

#get_classifiers(params = {}) ⇒ Types::GetClassifiersResponse

Lists all classifier objects in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.get_classifiers({
  max_results: 1,
  next_token: "Token",
})

Response structure


resp.classifiers #=> Array
resp.classifiers[0].grok_classifier.name #=> String
resp.classifiers[0].grok_classifier.classification #=> String
resp.classifiers[0].grok_classifier.creation_time #=> Time
resp.classifiers[0].grok_classifier.last_updated #=> Time
resp.classifiers[0].grok_classifier.version #=> Integer
resp.classifiers[0].grok_classifier.grok_pattern #=> String
resp.classifiers[0].grok_classifier.custom_patterns #=> String
resp.classifiers[0].xml_classifier.name #=> String
resp.classifiers[0].xml_classifier.classification #=> String
resp.classifiers[0].xml_classifier.creation_time #=> Time
resp.classifiers[0].xml_classifier.last_updated #=> Time
resp.classifiers[0].xml_classifier.version #=> Integer
resp.classifiers[0].xml_classifier.row_tag #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :max_results (Integer)

    Size of the list to return (optional).

  • :next_token (String)

    An optional continuation token.

Returns:

See Also:



1609
1610
1611
1612
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1609

def get_classifiers(params = {}, options = {})
  req = build_request(:get_classifiers, params)
  req.send_request(options)
end

#get_connection(params = {}) ⇒ Types::GetConnectionResponse

Retrieves a connection definition from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.get_connection({
  catalog_id: "CatalogIdString",
  name: "NameString", # required
})

Response structure


resp.connection.name #=> String
resp.connection.description #=> String
resp.connection.connection_type #=> String, one of "JDBC", "SFTP"
resp.connection.match_criteria #=> Array
resp.connection.match_criteria[0] #=> String
resp.connection.connection_properties #=> Hash
resp.connection.connection_properties["ConnectionPropertyKey"] #=> String
resp.connection.physical_connection_requirements.subnet_id #=> String
resp.connection.physical_connection_requirements.security_group_id_list #=> Array
resp.connection.physical_connection_requirements.security_group_id_list[0] #=> String
resp.connection.physical_connection_requirements.availability_zone #=> String
resp.connection.creation_time #=> Time
resp.connection.last_updated_time #=> Time
resp.connection.last_updated_by #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the connection resides. If none is supplied, the AWS account ID is used by default.

  • :name (required, String)

    The name of the connection definition to retrieve.

Returns:

See Also:



1655
1656
1657
1658
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1655

def get_connection(params = {}, options = {})
  req = build_request(:get_connection, params)
  req.send_request(options)
end

#get_connections(params = {}) ⇒ Types::GetConnectionsResponse

Retrieves a list of connection definitions from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.get_connections({
  catalog_id: "CatalogIdString",
  filter: {
    match_criteria: ["NameString"],
    connection_type: "JDBC", # accepts JDBC, SFTP
  },
  next_token: "Token",
  max_results: 1,
})

Response structure


resp.connection_list #=> Array
resp.connection_list[0].name #=> String
resp.connection_list[0].description #=> String
resp.connection_list[0].connection_type #=> String, one of "JDBC", "SFTP"
resp.connection_list[0].match_criteria #=> Array
resp.connection_list[0].match_criteria[0] #=> String
resp.connection_list[0].connection_properties #=> Hash
resp.connection_list[0].connection_properties["ConnectionPropertyKey"] #=> String
resp.connection_list[0].physical_connection_requirements.subnet_id #=> String
resp.connection_list[0].physical_connection_requirements.security_group_id_list #=> Array
resp.connection_list[0].physical_connection_requirements.security_group_id_list[0] #=> String
resp.connection_list[0].physical_connection_requirements.availability_zone #=> String
resp.connection_list[0].creation_time #=> Time
resp.connection_list[0].last_updated_time #=> Time
resp.connection_list[0].last_updated_by #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the connections reside. If none is supplied, the AWS account ID is used by default.

  • :filter (Types::GetConnectionsFilter)

    A filter that controls which connections will be returned.

  • :next_token (String)

    A continuation token, if this is a continuation call.

  • :max_results (Integer)

    The maximum number of connections to return in one response.

Returns:

See Also:



1715
1716
1717
1718
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1715

def get_connections(params = {}, options = {})
  req = build_request(:get_connections, params)
  req.send_request(options)
end

#get_crawler(params = {}) ⇒ Types::GetCrawlerResponse

Retrieves metadata for a specified crawler.

Examples:

Request syntax with placeholder values


resp = client.get_crawler({
  name: "NameString", # required
})

Response structure


resp.crawler.name #=> String
resp.crawler.role #=> String
resp.crawler.targets.s3_targets #=> Array
resp.crawler.targets.s3_targets[0].path #=> String
resp.crawler.targets.s3_targets[0].exclusions #=> Array
resp.crawler.targets.s3_targets[0].exclusions[0] #=> String
resp.crawler.targets.jdbc_targets #=> Array
resp.crawler.targets.jdbc_targets[0].connection_name #=> String
resp.crawler.targets.jdbc_targets[0].path #=> String
resp.crawler.targets.jdbc_targets[0].exclusions #=> Array
resp.crawler.targets.jdbc_targets[0].exclusions[0] #=> String
resp.crawler.database_name #=> String
resp.crawler.description #=> String
resp.crawler.classifiers #=> Array
resp.crawler.classifiers[0] #=> String
resp.crawler.schema_change_policy.update_behavior #=> String, one of "LOG", "UPDATE_IN_DATABASE"
resp.crawler.schema_change_policy.delete_behavior #=> String, one of "LOG", "DELETE_FROM_DATABASE", "DEPRECATE_IN_DATABASE"
resp.crawler.state #=> String, one of "READY", "RUNNING", "STOPPING"
resp.crawler.table_prefix #=> String
resp.crawler.schedule.schedule_expression #=> String
resp.crawler.schedule.state #=> String, one of "SCHEDULED", "NOT_SCHEDULED", "TRANSITIONING"
resp.crawler.crawl_elapsed_time #=> Integer
resp.crawler.creation_time #=> Time
resp.crawler.last_updated #=> Time
resp.crawler.last_crawl.status #=> String, one of "SUCCEEDED", "CANCELLED", "FAILED"
resp.crawler.last_crawl.error_message #=> String
resp.crawler.last_crawl.log_group #=> String
resp.crawler.last_crawl.log_stream #=> String
resp.crawler.last_crawl.message_prefix #=> String
resp.crawler.last_crawl.start_time #=> Time
resp.crawler.version #=> Integer
resp.crawler.configuration #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the crawler to retrieve metadata for.

Returns:

See Also:



1774
1775
1776
1777
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1774

def get_crawler(params = {}, options = {})
  req = build_request(:get_crawler, params)
  req.send_request(options)
end

#get_crawler_metrics(params = {}) ⇒ Types::GetCrawlerMetricsResponse

Retrieves metrics about specified crawlers.

Examples:

Request syntax with placeholder values


resp = client.get_crawler_metrics({
  crawler_name_list: ["NameString"],
  max_results: 1,
  next_token: "Token",
})

Response structure


resp.crawler_metrics_list #=> Array
resp.crawler_metrics_list[0].crawler_name #=> String
resp.crawler_metrics_list[0].time_left_seconds #=> Float
resp.crawler_metrics_list[0].still_estimating #=> Boolean
resp.crawler_metrics_list[0].last_runtime_seconds #=> Float
resp.crawler_metrics_list[0].median_runtime_seconds #=> Float
resp.crawler_metrics_list[0].tables_created #=> Integer
resp.crawler_metrics_list[0].tables_updated #=> Integer
resp.crawler_metrics_list[0].tables_deleted #=> Integer
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :crawler_name_list (Array<String>)

    A list of the names of crawlers about which to retrieve metrics.

  • :max_results (Integer)

    The maximum size of a list to return.

  • :next_token (String)

    A continuation token, if this is a continuation call.

Returns:

See Also:



1820
1821
1822
1823
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1820

def get_crawler_metrics(params = {}, options = {})
  req = build_request(:get_crawler_metrics, params)
  req.send_request(options)
end

#get_crawlers(params = {}) ⇒ Types::GetCrawlersResponse

Retrieves metadata for all crawlers defined in the customer account.

Examples:

Request syntax with placeholder values


resp = client.get_crawlers({
  max_results: 1,
  next_token: "Token",
})

Response structure


resp.crawlers #=> Array
resp.crawlers[0].name #=> String
resp.crawlers[0].role #=> String
resp.crawlers[0].targets.s3_targets #=> Array
resp.crawlers[0].targets.s3_targets[0].path #=> String
resp.crawlers[0].targets.s3_targets[0].exclusions #=> Array
resp.crawlers[0].targets.s3_targets[0].exclusions[0] #=> String
resp.crawlers[0].targets.jdbc_targets #=> Array
resp.crawlers[0].targets.jdbc_targets[0].connection_name #=> String
resp.crawlers[0].targets.jdbc_targets[0].path #=> String
resp.crawlers[0].targets.jdbc_targets[0].exclusions #=> Array
resp.crawlers[0].targets.jdbc_targets[0].exclusions[0] #=> String
resp.crawlers[0].database_name #=> String
resp.crawlers[0].description #=> String
resp.crawlers[0].classifiers #=> Array
resp.crawlers[0].classifiers[0] #=> String
resp.crawlers[0].schema_change_policy.update_behavior #=> String, one of "LOG", "UPDATE_IN_DATABASE"
resp.crawlers[0].schema_change_policy.delete_behavior #=> String, one of "LOG", "DELETE_FROM_DATABASE", "DEPRECATE_IN_DATABASE"
resp.crawlers[0].state #=> String, one of "READY", "RUNNING", "STOPPING"
resp.crawlers[0].table_prefix #=> String
resp.crawlers[0].schedule.schedule_expression #=> String
resp.crawlers[0].schedule.state #=> String, one of "SCHEDULED", "NOT_SCHEDULED", "TRANSITIONING"
resp.crawlers[0].crawl_elapsed_time #=> Integer
resp.crawlers[0].creation_time #=> Time
resp.crawlers[0].last_updated #=> Time
resp.crawlers[0].last_crawl.status #=> String, one of "SUCCEEDED", "CANCELLED", "FAILED"
resp.crawlers[0].last_crawl.error_message #=> String
resp.crawlers[0].last_crawl.log_group #=> String
resp.crawlers[0].last_crawl.log_stream #=> String
resp.crawlers[0].last_crawl.message_prefix #=> String
resp.crawlers[0].last_crawl.start_time #=> Time
resp.crawlers[0].version #=> Integer
resp.crawlers[0].configuration #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :max_results (Integer)

    The number of crawlers to return on each call.

  • :next_token (String)

    A continuation token, if this is a continuation request.

Returns:

See Also:



1886
1887
1888
1889
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1886

def get_crawlers(params = {}, options = {})
  req = build_request(:get_crawlers, params)
  req.send_request(options)
end

#get_database(params = {}) ⇒ Types::GetDatabaseResponse

Retrieves the definition of a specified database.

Examples:

Request syntax with placeholder values


resp = client.get_database({
  catalog_id: "CatalogIdString",
  name: "NameString", # required
})

Response structure


resp.database.name #=> String
resp.database.description #=> String
resp.database.location_uri #=> String
resp.database.parameters #=> Hash
resp.database.parameters["KeyString"] #=> String
resp.database.create_time #=> Time

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the database resides. If none is supplied, the AWS account ID is used by default.

  • :name (required, String)

    The name of the database to retrieve.

Returns:

See Also:



1924
1925
1926
1927
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1924

def get_database(params = {}, options = {})
  req = build_request(:get_database, params)
  req.send_request(options)
end

#get_databases(params = {}) ⇒ Types::GetDatabasesResponse

Retrieves all Databases defined in a given Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.get_databases({
  catalog_id: "CatalogIdString",
  next_token: "Token",
  max_results: 1,
})

Response structure


resp.database_list #=> Array
resp.database_list[0].name #=> String
resp.database_list[0].description #=> String
resp.database_list[0].location_uri #=> String
resp.database_list[0].parameters #=> Hash
resp.database_list[0].parameters["KeyString"] #=> String
resp.database_list[0].create_time #=> Time
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog from which to retrieve Databases. If none is supplied, the AWS account ID is used by default.

  • :next_token (String)

    A continuation token, if this is a continuation call.

  • :max_results (Integer)

    The maximum number of databases to return in one response.

Returns:

See Also:



1969
1970
1971
1972
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 1969

def get_databases(params = {}, options = {})
  req = build_request(:get_databases, params)
  req.send_request(options)
end

#get_dataflow_graph(params = {}) ⇒ Types::GetDataflowGraphResponse

Transforms a Python script into a directed acyclic graph (DAG).

Examples:

Request syntax with placeholder values


resp = client.get_dataflow_graph({
  python_script: "PythonScript",
})

Response structure


resp.dag_nodes #=> Array
resp.dag_nodes[0].id #=> String
resp.dag_nodes[0].node_type #=> String
resp.dag_nodes[0].args #=> Array
resp.dag_nodes[0].args[0].name #=> String
resp.dag_nodes[0].args[0].value #=> String
resp.dag_nodes[0].args[0].param #=> Boolean
resp.dag_nodes[0].line_number #=> Integer
resp.dag_edges #=> Array
resp.dag_edges[0].source #=> String
resp.dag_edges[0].target #=> String
resp.dag_edges[0].target_parameter #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :python_script (String)

    The Python script to transform.

Returns:

See Also:



2009
2010
2011
2012
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2009

def get_dataflow_graph(params = {}, options = {})
  req = build_request(:get_dataflow_graph, params)
  req.send_request(options)
end

#get_dev_endpoint(params = {}) ⇒ Types::GetDevEndpointResponse

Retrieves information about a specified DevEndpoint.

Examples:

Request syntax with placeholder values


resp = client.get_dev_endpoint({
  endpoint_name: "GenericString", # required
})

Response structure


resp.dev_endpoint.endpoint_name #=> String
resp.dev_endpoint.role_arn #=> String
resp.dev_endpoint.security_group_ids #=> Array
resp.dev_endpoint.security_group_ids[0] #=> String
resp.dev_endpoint.subnet_id #=> String
resp.dev_endpoint.yarn_endpoint_address #=> String
resp.dev_endpoint.zeppelin_remote_spark_interpreter_port #=> Integer
resp.dev_endpoint.public_address #=> String
resp.dev_endpoint.status #=> String
resp.dev_endpoint.number_of_nodes #=> Integer
resp.dev_endpoint.availability_zone #=> String
resp.dev_endpoint.vpc_id #=> String
resp.dev_endpoint.extra_python_libs_s3_path #=> String
resp.dev_endpoint.extra_jars_s3_path #=> String
resp.dev_endpoint.failure_reason #=> String
resp.dev_endpoint.last_update_status #=> String
resp.dev_endpoint.created_timestamp #=> Time
resp.dev_endpoint.last_modified_timestamp #=> Time
resp.dev_endpoint.public_key #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :endpoint_name (required, String)

    Name of the DevEndpoint for which to retrieve information.

Returns:

See Also:



2055
2056
2057
2058
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2055

def get_dev_endpoint(params = {}, options = {})
  req = build_request(:get_dev_endpoint, params)
  req.send_request(options)
end

#get_dev_endpoints(params = {}) ⇒ Types::GetDevEndpointsResponse

Retrieves all the DevEndpoints in this AWS account.

Examples:

Request syntax with placeholder values


resp = client.get_dev_endpoints({
  max_results: 1,
  next_token: "GenericString",
})

Response structure


resp.dev_endpoints #=> Array
resp.dev_endpoints[0].endpoint_name #=> String
resp.dev_endpoints[0].role_arn #=> String
resp.dev_endpoints[0].security_group_ids #=> Array
resp.dev_endpoints[0].security_group_ids[0] #=> String
resp.dev_endpoints[0].subnet_id #=> String
resp.dev_endpoints[0].yarn_endpoint_address #=> String
resp.dev_endpoints[0].zeppelin_remote_spark_interpreter_port #=> Integer
resp.dev_endpoints[0].public_address #=> String
resp.dev_endpoints[0].status #=> String
resp.dev_endpoints[0].number_of_nodes #=> Integer
resp.dev_endpoints[0].availability_zone #=> String
resp.dev_endpoints[0].vpc_id #=> String
resp.dev_endpoints[0].extra_python_libs_s3_path #=> String
resp.dev_endpoints[0].extra_jars_s3_path #=> String
resp.dev_endpoints[0].failure_reason #=> String
resp.dev_endpoints[0].last_update_status #=> String
resp.dev_endpoints[0].created_timestamp #=> Time
resp.dev_endpoints[0].last_modified_timestamp #=> Time
resp.dev_endpoints[0].public_key #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :max_results (Integer)

    The maximum size of information to return.

  • :next_token (String)

    A continuation token, if this is a continuation call.

Returns:

See Also:



2108
2109
2110
2111
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2108

def get_dev_endpoints(params = {}, options = {})
  req = build_request(:get_dev_endpoints, params)
  req.send_request(options)
end

#get_job(params = {}) ⇒ Types::GetJobResponse

Retrieves an existing job definition.

Examples:

Request syntax with placeholder values


resp = client.get_job({
  job_name: "NameString", # required
})

Response structure


resp.job.name #=> String
resp.job.description #=> String
resp.job.log_uri #=> String
resp.job.role #=> String
resp.job.created_on #=> Time
resp.job.last_modified_on #=> Time
resp.job.execution_property.max_concurrent_runs #=> Integer
resp.job.command.name #=> String
resp.job.command.script_location #=> String
resp.job.default_arguments #=> Hash
resp.job.default_arguments["GenericString"] #=> String
resp.job.connections.connections #=> Array
resp.job.connections.connections[0] #=> String
resp.job.max_retries #=> Integer
resp.job.allocated_capacity #=> Integer

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    The name of the job to retrieve.

Returns:

See Also:



2150
2151
2152
2153
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2150

def get_job(params = {}, options = {})
  req = build_request(:get_job, params)
  req.send_request(options)
end

#get_job_run(params = {}) ⇒ Types::GetJobRunResponse

Retrieves the metadata for a given job run.

Examples:

Request syntax with placeholder values


resp = client.get_job_run({
  job_name: "NameString", # required
  run_id: "IdString", # required
  predecessors_included: false,
})

Response structure


resp.job_run.id #=> String
resp.job_run.attempt #=> Integer
resp.job_run.previous_run_id #=> String
resp.job_run.trigger_name #=> String
resp.job_run.job_name #=> String
resp.job_run.started_on #=> Time
resp.job_run.last_modified_on #=> Time
resp.job_run.completed_on #=> Time
resp.job_run.job_run_state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED"
resp.job_run.arguments #=> Hash
resp.job_run.arguments["GenericString"] #=> String
resp.job_run.error_message #=> String
resp.job_run.predecessor_runs #=> Array
resp.job_run.predecessor_runs[0].job_name #=> String
resp.job_run.predecessor_runs[0].run_id #=> String
resp.job_run.allocated_capacity #=> Integer

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    Name of the job being run.

  • :run_id (required, String)

    The ID of the job run.

  • :predecessors_included (Boolean)

    A list of the predecessor runs to return as well.

Returns:

See Also:



2201
2202
2203
2204
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2201

def get_job_run(params = {}, options = {})
  req = build_request(:get_job_run, params)
  req.send_request(options)
end

#get_job_runs(params = {}) ⇒ Types::GetJobRunsResponse

Retrieves metadata for all runs of a given job.

Examples:

Request syntax with placeholder values


resp = client.get_job_runs({
  job_name: "NameString", # required
  next_token: "GenericString",
  max_results: 1,
})

Response structure


resp.job_runs #=> Array
resp.job_runs[0].id #=> String
resp.job_runs[0].attempt #=> Integer
resp.job_runs[0].previous_run_id #=> String
resp.job_runs[0].trigger_name #=> String
resp.job_runs[0].job_name #=> String
resp.job_runs[0].started_on #=> Time
resp.job_runs[0].last_modified_on #=> Time
resp.job_runs[0].completed_on #=> Time
resp.job_runs[0].job_run_state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED"
resp.job_runs[0].arguments #=> Hash
resp.job_runs[0].arguments["GenericString"] #=> String
resp.job_runs[0].error_message #=> String
resp.job_runs[0].predecessor_runs #=> Array
resp.job_runs[0].predecessor_runs[0].job_name #=> String
resp.job_runs[0].predecessor_runs[0].run_id #=> String
resp.job_runs[0].allocated_capacity #=> Integer
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    The name of the job for which to retrieve all job runs.

  • :next_token (String)

    A continuation token, if this is a continuation call.

  • :max_results (Integer)

    The maximum size of the response.

Returns:

See Also:



2255
2256
2257
2258
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2255

def get_job_runs(params = {}, options = {})
  req = build_request(:get_job_runs, params)
  req.send_request(options)
end

#get_jobs(params = {}) ⇒ Types::GetJobsResponse

Retrieves all current jobs.

Examples:

Request syntax with placeholder values


resp = client.get_jobs({
  next_token: "GenericString",
  max_results: 1,
})

Response structure


resp.jobs #=> Array
resp.jobs[0].name #=> String
resp.jobs[0].description #=> String
resp.jobs[0].log_uri #=> String
resp.jobs[0].role #=> String
resp.jobs[0].created_on #=> Time
resp.jobs[0].last_modified_on #=> Time
resp.jobs[0].execution_property.max_concurrent_runs #=> Integer
resp.jobs[0].command.name #=> String
resp.jobs[0].command.script_location #=> String
resp.jobs[0].default_arguments #=> Hash
resp.jobs[0].default_arguments["GenericString"] #=> String
resp.jobs[0].connections.connections #=> Array
resp.jobs[0].connections.connections[0] #=> String
resp.jobs[0].max_retries #=> Integer
resp.jobs[0].allocated_capacity #=> Integer
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :next_token (String)

    A continuation token, if this is a continuation call.

  • :max_results (Integer)

    The maximum size of the response.

Returns:

See Also:



2304
2305
2306
2307
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2304

def get_jobs(params = {}, options = {})
  req = build_request(:get_jobs, params)
  req.send_request(options)
end

#get_mapping(params = {}) ⇒ Types::GetMappingResponse

Creates mappings.

Examples:

Request syntax with placeholder values


resp = client.get_mapping({
  source: { # required
    database_name: "NameString", # required
    table_name: "NameString", # required
  },
  sinks: [
    {
      database_name: "NameString", # required
      table_name: "NameString", # required
    },
  ],
  location: {
    jdbc: [
      {
        name: "CodeGenArgName", # required
        value: "CodeGenArgValue", # required
        param: false,
      },
    ],
    s3: [
      {
        name: "CodeGenArgName", # required
        value: "CodeGenArgValue", # required
        param: false,
      },
    ],
  },
})

Response structure


resp.mapping #=> Array
resp.mapping[0].source_table #=> String
resp.mapping[0].source_path #=> String
resp.mapping[0].source_type #=> String
resp.mapping[0].target_table #=> String
resp.mapping[0].target_path #=> String
resp.mapping[0].target_type #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

Returns:

See Also:



2369
2370
2371
2372
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2369

def get_mapping(params = {}, options = {})
  req = build_request(:get_mapping, params)
  req.send_request(options)
end

#get_partition(params = {}) ⇒ Types::GetPartitionResponse

Retrieves information about a specified partition.

Examples:

Request syntax with placeholder values


resp = client.get_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partition_values: ["ValueString"], # required
})

Response structure


resp.partition.values #=> Array
resp.partition.values[0] #=> String
resp.partition.database_name #=> String
resp.partition.table_name #=> String
resp.partition.creation_time #=> Time
resp.partition.last_access_time #=> Time
resp.partition.storage_descriptor.columns #=> Array
resp.partition.storage_descriptor.columns[0].name #=> String
resp.partition.storage_descriptor.columns[0].type #=> String
resp.partition.storage_descriptor.columns[0].comment #=> String
resp.partition.storage_descriptor.location #=> String
resp.partition.storage_descriptor.input_format #=> String
resp.partition.storage_descriptor.output_format #=> String
resp.partition.storage_descriptor.compressed #=> Boolean
resp.partition.storage_descriptor.number_of_buckets #=> Integer
resp.partition.storage_descriptor.serde_info.name #=> String
resp.partition.storage_descriptor.serde_info.serialization_library #=> String
resp.partition.storage_descriptor.serde_info.parameters #=> Hash
resp.partition.storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.partition.storage_descriptor.bucket_columns #=> Array
resp.partition.storage_descriptor.bucket_columns[0] #=> String
resp.partition.storage_descriptor.sort_columns #=> Array
resp.partition.storage_descriptor.sort_columns[0].column #=> String
resp.partition.storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.partition.storage_descriptor.parameters #=> Hash
resp.partition.storage_descriptor.parameters["KeyString"] #=> String
resp.partition.storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.partition.storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.partition.storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.partition.storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.partition.storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.partition.storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.partition.storage_descriptor.stored_as_sub_directories #=> Boolean
resp.partition.parameters #=> Hash
resp.partition.parameters["KeyString"] #=> String
resp.partition.last_analyzed_time #=> Time

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the partition in question resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the partition resides.

  • :table_name (required, String)

    The name of the partition's table.

  • :partition_values (required, Array<String>)

    The values that define the partition.

Returns:

See Also:



2445
2446
2447
2448
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2445

def get_partition(params = {}, options = {})
  req = build_request(:get_partition, params)
  req.send_request(options)
end

#get_partitions(params = {}) ⇒ Types::GetPartitionsResponse

Retrieves information about the partitions in a table.

Examples:

Request syntax with placeholder values


resp = client.get_partitions({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  expression: "PredicateString",
  next_token: "Token",
  segment: {
    segment_number: 1, # required
    total_segments: 1, # required
  },
  max_results: 1,
})

Response structure


resp.partitions #=> Array
resp.partitions[0].values #=> Array
resp.partitions[0].values[0] #=> String
resp.partitions[0].database_name #=> String
resp.partitions[0].table_name #=> String
resp.partitions[0].creation_time #=> Time
resp.partitions[0].last_access_time #=> Time
resp.partitions[0].storage_descriptor.columns #=> Array
resp.partitions[0].storage_descriptor.columns[0].name #=> String
resp.partitions[0].storage_descriptor.columns[0].type #=> String
resp.partitions[0].storage_descriptor.columns[0].comment #=> String
resp.partitions[0].storage_descriptor.location #=> String
resp.partitions[0].storage_descriptor.input_format #=> String
resp.partitions[0].storage_descriptor.output_format #=> String
resp.partitions[0].storage_descriptor.compressed #=> Boolean
resp.partitions[0].storage_descriptor.number_of_buckets #=> Integer
resp.partitions[0].storage_descriptor.serde_info.name #=> String
resp.partitions[0].storage_descriptor.serde_info.serialization_library #=> String
resp.partitions[0].storage_descriptor.serde_info.parameters #=> Hash
resp.partitions[0].storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.bucket_columns #=> Array
resp.partitions[0].storage_descriptor.bucket_columns[0] #=> String
resp.partitions[0].storage_descriptor.sort_columns #=> Array
resp.partitions[0].storage_descriptor.sort_columns[0].column #=> String
resp.partitions[0].storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.partitions[0].storage_descriptor.parameters #=> Hash
resp.partitions[0].storage_descriptor.parameters["KeyString"] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.partitions[0].storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.partitions[0].storage_descriptor.stored_as_sub_directories #=> Boolean
resp.partitions[0].parameters #=> Hash
resp.partitions[0].parameters["KeyString"] #=> String
resp.partitions[0].last_analyzed_time #=> Time
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the partitions in question reside. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the partitions reside.

  • :table_name (required, String)

    The name of the partitions' table.

  • :expression (String)

    An expression filtering the partitions to be returned.

  • :next_token (String)

    A continuation token, if this is not the first call to retrieve these partitions.

  • :segment (Types::Segment)

    The segment of the table's partitions to scan in this request.

  • :max_results (Integer)

    The maximum number of partitions to return in a single response.

Returns:

See Also:



2540
2541
2542
2543
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2540

def get_partitions(params = {}, options = {})
  req = build_request(:get_partitions, params)
  req.send_request(options)
end

#get_plan(params = {}) ⇒ Types::GetPlanResponse

Gets a Python script to perform a specified mapping.

Examples:

Request syntax with placeholder values


resp = client.get_plan({
  mapping: [ # required
    {
      source_table: "TableName",
      source_path: "SchemaPathString",
      source_type: "FieldType",
      target_table: "TableName",
      target_path: "SchemaPathString",
      target_type: "FieldType",
    },
  ],
  source: { # required
    database_name: "NameString", # required
    table_name: "NameString", # required
  },
  sinks: [
    {
      database_name: "NameString", # required
      table_name: "NameString", # required
    },
  ],
  location: {
    jdbc: [
      {
        name: "CodeGenArgName", # required
        value: "CodeGenArgValue", # required
        param: false,
      },
    ],
    s3: [
      {
        name: "CodeGenArgName", # required
        value: "CodeGenArgValue", # required
        param: false,
      },
    ],
  },
})

Response structure


resp.python_script #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

Returns:

See Also:



2612
2613
2614
2615
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2612

def get_plan(params = {}, options = {})
  req = build_request(:get_plan, params)
  req.send_request(options)
end

#get_table(params = {}) ⇒ Types::GetTableResponse

Retrieves the Table definition in a Data Catalog for a specified table.

Examples:

Request syntax with placeholder values


resp = client.get_table({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  name: "NameString", # required
})

Response structure


resp.table.name #=> String
resp.table.database_name #=> String
resp.table.description #=> String
resp.table.owner #=> String
resp.table.create_time #=> Time
resp.table.update_time #=> Time
resp.table.last_access_time #=> Time
resp.table.last_analyzed_time #=> Time
resp.table.retention #=> Integer
resp.table.storage_descriptor.columns #=> Array
resp.table.storage_descriptor.columns[0].name #=> String
resp.table.storage_descriptor.columns[0].type #=> String
resp.table.storage_descriptor.columns[0].comment #=> String
resp.table.storage_descriptor.location #=> String
resp.table.storage_descriptor.input_format #=> String
resp.table.storage_descriptor.output_format #=> String
resp.table.storage_descriptor.compressed #=> Boolean
resp.table.storage_descriptor.number_of_buckets #=> Integer
resp.table.storage_descriptor.serde_info.name #=> String
resp.table.storage_descriptor.serde_info.serialization_library #=> String
resp.table.storage_descriptor.serde_info.parameters #=> Hash
resp.table.storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.table.storage_descriptor.bucket_columns #=> Array
resp.table.storage_descriptor.bucket_columns[0] #=> String
resp.table.storage_descriptor.sort_columns #=> Array
resp.table.storage_descriptor.sort_columns[0].column #=> String
resp.table.storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.table.storage_descriptor.parameters #=> Hash
resp.table.storage_descriptor.parameters["KeyString"] #=> String
resp.table.storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.table.storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.table.storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.table.storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.table.storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.table.storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.table.storage_descriptor.stored_as_sub_directories #=> Boolean
resp.table.partition_keys #=> Array
resp.table.partition_keys[0].name #=> String
resp.table.partition_keys[0].type #=> String
resp.table.partition_keys[0].comment #=> String
resp.table.view_original_text #=> String
resp.table.view_expanded_text #=> String
resp.table.table_type #=> String
resp.table.parameters #=> Hash
resp.table.parameters["KeyString"] #=> String
resp.table.created_by #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the database in the catalog in which the table resides.

  • :name (required, String)

    The name of the table for which to retrieve the definition.

Returns:

See Also:



2695
2696
2697
2698
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2695

def get_table(params = {}, options = {})
  req = build_request(:get_table, params)
  req.send_request(options)
end

#get_table_versions(params = {}) ⇒ Types::GetTableVersionsResponse

Retrieves a list of strings that identify available versions of a specified table.

Examples:

Request syntax with placeholder values


resp = client.get_table_versions({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  next_token: "Token",
  max_results: 1,
})

Response structure


resp.table_versions #=> Array
resp.table_versions[0].table.name #=> String
resp.table_versions[0].table.database_name #=> String
resp.table_versions[0].table.description #=> String
resp.table_versions[0].table.owner #=> String
resp.table_versions[0].table.create_time #=> Time
resp.table_versions[0].table.update_time #=> Time
resp.table_versions[0].table.last_access_time #=> Time
resp.table_versions[0].table.last_analyzed_time #=> Time
resp.table_versions[0].table.retention #=> Integer
resp.table_versions[0].table.storage_descriptor.columns #=> Array
resp.table_versions[0].table.storage_descriptor.columns[0].name #=> String
resp.table_versions[0].table.storage_descriptor.columns[0].type #=> String
resp.table_versions[0].table.storage_descriptor.columns[0].comment #=> String
resp.table_versions[0].table.storage_descriptor.location #=> String
resp.table_versions[0].table.storage_descriptor.input_format #=> String
resp.table_versions[0].table.storage_descriptor.output_format #=> String
resp.table_versions[0].table.storage_descriptor.compressed #=> Boolean
resp.table_versions[0].table.storage_descriptor.number_of_buckets #=> Integer
resp.table_versions[0].table.storage_descriptor.serde_info.name #=> String
resp.table_versions[0].table.storage_descriptor.serde_info.serialization_library #=> String
resp.table_versions[0].table.storage_descriptor.serde_info.parameters #=> Hash
resp.table_versions[0].table.storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.table_versions[0].table.storage_descriptor.bucket_columns #=> Array
resp.table_versions[0].table.storage_descriptor.bucket_columns[0] #=> String
resp.table_versions[0].table.storage_descriptor.sort_columns #=> Array
resp.table_versions[0].table.storage_descriptor.sort_columns[0].column #=> String
resp.table_versions[0].table.storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.table_versions[0].table.storage_descriptor.parameters #=> Hash
resp.table_versions[0].table.storage_descriptor.parameters["KeyString"] #=> String
resp.table_versions[0].table.storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.table_versions[0].table.storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.table_versions[0].table.storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.table_versions[0].table.storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.table_versions[0].table.storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.table_versions[0].table.storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.table_versions[0].table.storage_descriptor.stored_as_sub_directories #=> Boolean
resp.table_versions[0].table.partition_keys #=> Array
resp.table_versions[0].table.partition_keys[0].name #=> String
resp.table_versions[0].table.partition_keys[0].type #=> String
resp.table_versions[0].table.partition_keys[0].comment #=> String
resp.table_versions[0].table.view_original_text #=> String
resp.table_versions[0].table.view_expanded_text #=> String
resp.table_versions[0].table.table_type #=> String
resp.table_versions[0].table.parameters #=> Hash
resp.table_versions[0].table.parameters["KeyString"] #=> String
resp.table_versions[0].table.created_by #=> String
resp.table_versions[0].version_id #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The database in the catalog in which the table resides.

  • :table_name (required, String)

    The name of the table.

  • :next_token (String)

    A continuation token, if this is not the first call.

  • :max_results (Integer)

    The maximum number of table versions to return in one response.

Returns:

See Also:



2790
2791
2792
2793
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2790

def get_table_versions(params = {}, options = {})
  req = build_request(:get_table_versions, params)
  req.send_request(options)
end

#get_tables(params = {}) ⇒ Types::GetTablesResponse

Retrieves the definitions of some or all of the tables in a given Database.

Examples:

Request syntax with placeholder values


resp = client.get_tables({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  expression: "FilterString",
  next_token: "Token",
  max_results: 1,
})

Response structure


resp.table_list #=> Array
resp.table_list[0].name #=> String
resp.table_list[0].database_name #=> String
resp.table_list[0].description #=> String
resp.table_list[0].owner #=> String
resp.table_list[0].create_time #=> Time
resp.table_list[0].update_time #=> Time
resp.table_list[0].last_access_time #=> Time
resp.table_list[0].last_analyzed_time #=> Time
resp.table_list[0].retention #=> Integer
resp.table_list[0].storage_descriptor.columns #=> Array
resp.table_list[0].storage_descriptor.columns[0].name #=> String
resp.table_list[0].storage_descriptor.columns[0].type #=> String
resp.table_list[0].storage_descriptor.columns[0].comment #=> String
resp.table_list[0].storage_descriptor.location #=> String
resp.table_list[0].storage_descriptor.input_format #=> String
resp.table_list[0].storage_descriptor.output_format #=> String
resp.table_list[0].storage_descriptor.compressed #=> Boolean
resp.table_list[0].storage_descriptor.number_of_buckets #=> Integer
resp.table_list[0].storage_descriptor.serde_info.name #=> String
resp.table_list[0].storage_descriptor.serde_info.serialization_library #=> String
resp.table_list[0].storage_descriptor.serde_info.parameters #=> Hash
resp.table_list[0].storage_descriptor.serde_info.parameters["KeyString"] #=> String
resp.table_list[0].storage_descriptor.bucket_columns #=> Array
resp.table_list[0].storage_descriptor.bucket_columns[0] #=> String
resp.table_list[0].storage_descriptor.sort_columns #=> Array
resp.table_list[0].storage_descriptor.sort_columns[0].column #=> String
resp.table_list[0].storage_descriptor.sort_columns[0].sort_order #=> Integer
resp.table_list[0].storage_descriptor.parameters #=> Hash
resp.table_list[0].storage_descriptor.parameters["KeyString"] #=> String
resp.table_list[0].storage_descriptor.skewed_info.skewed_column_names #=> Array
resp.table_list[0].storage_descriptor.skewed_info.skewed_column_names[0] #=> String
resp.table_list[0].storage_descriptor.skewed_info.skewed_column_values #=> Array
resp.table_list[0].storage_descriptor.skewed_info.skewed_column_values[0] #=> String
resp.table_list[0].storage_descriptor.skewed_info.skewed_column_value_location_maps #=> Hash
resp.table_list[0].storage_descriptor.skewed_info.skewed_column_value_location_maps["ColumnValuesString"] #=> String
resp.table_list[0].storage_descriptor.stored_as_sub_directories #=> Boolean
resp.table_list[0].partition_keys #=> Array
resp.table_list[0].partition_keys[0].name #=> String
resp.table_list[0].partition_keys[0].type #=> String
resp.table_list[0].partition_keys[0].comment #=> String
resp.table_list[0].view_original_text #=> String
resp.table_list[0].view_expanded_text #=> String
resp.table_list[0].table_type #=> String
resp.table_list[0].parameters #=> Hash
resp.table_list[0].parameters["KeyString"] #=> String
resp.table_list[0].created_by #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the tables reside. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The database in the catalog whose tables to list.

  • :expression (String)

    A regular expression pattern. If present, only those tables whose names match the pattern are returned.

  • :next_token (String)

    A continuation token, included if this is a continuation call.

  • :max_results (Integer)

    The maximum number of tables to return in a single response.

Returns:

See Also:



2885
2886
2887
2888
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2885

def get_tables(params = {}, options = {})
  req = build_request(:get_tables, params)
  req.send_request(options)
end

#get_trigger(params = {}) ⇒ Types::GetTriggerResponse

Retrieves the definition of a trigger.

Examples:

Request syntax with placeholder values


resp = client.get_trigger({
  name: "NameString", # required
})

Response structure


resp.trigger.name #=> String
resp.trigger.id #=> String
resp.trigger.type #=> String, one of "SCHEDULED", "CONDITIONAL", "ON_DEMAND"
resp.trigger.state #=> String, one of "CREATING", "CREATED", "ACTIVATING", "ACTIVATED", "DEACTIVATING", "DEACTIVATED", "DELETING", "UPDATING"
resp.trigger.description #=> String
resp.trigger.schedule #=> String
resp.trigger.actions #=> Array
resp.trigger.actions[0].job_name #=> String
resp.trigger.actions[0].arguments #=> Hash
resp.trigger.actions[0].arguments["GenericString"] #=> String
resp.trigger.predicate.logical #=> String, one of "AND"
resp.trigger.predicate.conditions #=> Array
resp.trigger.predicate.conditions[0].logical_operator #=> String, one of "EQUALS"
resp.trigger.predicate.conditions[0].job_name #=> String
resp.trigger.predicate.conditions[0].state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED"

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name of the trigger to retrieve.

Returns:

See Also:



2927
2928
2929
2930
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2927

def get_trigger(params = {}, options = {})
  req = build_request(:get_trigger, params)
  req.send_request(options)
end

#get_triggers(params = {}) ⇒ Types::GetTriggersResponse

Gets all the triggers associated with a job.

Examples:

Request syntax with placeholder values


resp = client.get_triggers({
  next_token: "GenericString",
  dependent_job_name: "NameString",
  max_results: 1,
})

Response structure


resp.triggers #=> Array
resp.triggers[0].name #=> String
resp.triggers[0].id #=> String
resp.triggers[0].type #=> String, one of "SCHEDULED", "CONDITIONAL", "ON_DEMAND"
resp.triggers[0].state #=> String, one of "CREATING", "CREATED", "ACTIVATING", "ACTIVATED", "DEACTIVATING", "DEACTIVATED", "DELETING", "UPDATING"
resp.triggers[0].description #=> String
resp.triggers[0].schedule #=> String
resp.triggers[0].actions #=> Array
resp.triggers[0].actions[0].job_name #=> String
resp.triggers[0].actions[0].arguments #=> Hash
resp.triggers[0].actions[0].arguments["GenericString"] #=> String
resp.triggers[0].predicate.logical #=> String, one of "AND"
resp.triggers[0].predicate.conditions #=> Array
resp.triggers[0].predicate.conditions[0].logical_operator #=> String, one of "EQUALS"
resp.triggers[0].predicate.conditions[0].job_name #=> String
resp.triggers[0].predicate.conditions[0].state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED"
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :next_token (String)

    A continuation token, if this is a continuation call.

  • :dependent_job_name (String)

    The name of the job for which to retrieve triggers.

  • :max_results (Integer)

    The maximum size of the response.

Returns:

See Also:



2980
2981
2982
2983
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 2980

def get_triggers(params = {}, options = {})
  req = build_request(:get_triggers, params)
  req.send_request(options)
end

#get_user_defined_function(params = {}) ⇒ Types::GetUserDefinedFunctionResponse

Retrieves a specified function definition from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.get_user_defined_function({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  function_name: "NameString", # required
})

Response structure


resp.user_defined_function.function_name #=> String
resp.user_defined_function.class_name #=> String
resp.user_defined_function.owner_name #=> String
resp.user_defined_function.owner_type #=> String, one of "USER", "ROLE", "GROUP"
resp.user_defined_function.create_time #=> Time
resp.user_defined_function.resource_uris #=> Array
resp.user_defined_function.resource_uris[0].resource_type #=> String, one of "JAR", "FILE", "ARCHIVE"
resp.user_defined_function.resource_uris[0].uri #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the function to be retrieved is located. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the function is located.

  • :function_name (required, String)

    The name of the function.

Returns:

See Also:



3024
3025
3026
3027
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3024

def get_user_defined_function(params = {}, options = {})
  req = build_request(:get_user_defined_function, params)
  req.send_request(options)
end

#get_user_defined_functions(params = {}) ⇒ Types::GetUserDefinedFunctionsResponse

Retrieves a multiple function definitions from the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.get_user_defined_functions({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  pattern: "NameString", # required
  next_token: "Token",
  max_results: 1,
})

Response structure


resp.user_defined_functions #=> Array
resp.user_defined_functions[0].function_name #=> String
resp.user_defined_functions[0].class_name #=> String
resp.user_defined_functions[0].owner_name #=> String
resp.user_defined_functions[0].owner_type #=> String, one of "USER", "ROLE", "GROUP"
resp.user_defined_functions[0].create_time #=> Time
resp.user_defined_functions[0].resource_uris #=> Array
resp.user_defined_functions[0].resource_uris[0].resource_type #=> String, one of "JAR", "FILE", "ARCHIVE"
resp.user_defined_functions[0].resource_uris[0].uri #=> String
resp.next_token #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the functions to be retrieved are located. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the functions are located.

  • :pattern (required, String)

    An optional function-name pattern string that filters the function definitions returned.

  • :next_token (String)

    A continuation token, if this is a continuation call.

  • :max_results (Integer)

    The maximum number of functions to return in one response.

Returns:

See Also:



3080
3081
3082
3083
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3080

def get_user_defined_functions(params = {}, options = {})
  req = build_request(:get_user_defined_functions, params)
  req.send_request(options)
end

#import_catalog_to_glue(params = {}) ⇒ Struct

Imports an existing Athena Data Catalog to AWS Glue

Examples:

Request syntax with placeholder values


resp = client.import_catalog_to_glue({
  catalog_id: "CatalogIdString",
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the catalog to import. Currently, this should be the AWS account ID.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3103
3104
3105
3106
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3103

def import_catalog_to_glue(params = {}, options = {})
  req = build_request(:import_catalog_to_glue, params)
  req.send_request(options)
end

#reset_job_bookmark(params = {}) ⇒ Types::ResetJobBookmarkResponse

Resets a bookmark entry.

Examples:

Request syntax with placeholder values


resp = client.reset_job_bookmark({
  job_name: "JobName", # required
})

Response structure


resp.job_bookmark_entry.job_name #=> String
resp.job_bookmark_entry.version #=> Integer
resp.job_bookmark_entry.run #=> Integer
resp.job_bookmark_entry.attempt #=> Integer
resp.job_bookmark_entry.job_bookmark #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    The name of the job in question.

Returns:

See Also:



3135
3136
3137
3138
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3135

def reset_job_bookmark(params = {}, options = {})
  req = build_request(:reset_job_bookmark, params)
  req.send_request(options)
end

#start_crawler(params = {}) ⇒ Struct

Starts a crawl using the specified crawler, regardless of what is scheduled. If the crawler is already running, does nothing.

Examples:

Request syntax with placeholder values


resp = client.start_crawler({
  name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the crawler to start.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3158
3159
3160
3161
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3158

def start_crawler(params = {}, options = {})
  req = build_request(:start_crawler, params)
  req.send_request(options)
end

#start_crawler_schedule(params = {}) ⇒ Struct

Changes the schedule state of the specified crawler to SCHEDULED, unless the crawler is already running or the schedule state is already SCHEDULED.

Examples:

Request syntax with placeholder values


resp = client.start_crawler_schedule({
  crawler_name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :crawler_name (required, String)

    Name of the crawler to schedule.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3182
3183
3184
3185
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3182

def start_crawler_schedule(params = {}, options = {})
  req = build_request(:start_crawler_schedule, params)
  req.send_request(options)
end

#start_job_run(params = {}) ⇒ Types::StartJobRunResponse

Runs a job.

Examples:

Request syntax with placeholder values


resp = client.start_job_run({
  job_name: "NameString", # required
  job_run_id: "IdString",
  arguments: {
    "GenericString" => "GenericString",
  },
  allocated_capacity: 1,
})

Response structure


resp.job_run_id #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    The name of the job to start.

  • :job_run_id (String)

    The ID of the job run to start.

  • :arguments (Hash<String,String>)

    Specific arguments for this job run.

  • :allocated_capacity (Integer)

    The infrastructure capacity to allocate to this job.

Returns:

See Also:



3224
3225
3226
3227
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3224

def start_job_run(params = {}, options = {})
  req = build_request(:start_job_run, params)
  req.send_request(options)
end

#start_trigger(params = {}) ⇒ Types::StartTriggerResponse

Starts an existing trigger.

Examples:

Request syntax with placeholder values


resp = client.start_trigger({
  name: "NameString", # required
})

Response structure


resp.name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name of the trigger to start.

Returns:

See Also:



3252
3253
3254
3255
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3252

def start_trigger(params = {}, options = {})
  req = build_request(:start_trigger, params)
  req.send_request(options)
end

#stop_crawler(params = {}) ⇒ Struct

If the specified crawler is running, stops the crawl.

Examples:

Request syntax with placeholder values


resp = client.stop_crawler({
  name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the crawler to stop.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3274
3275
3276
3277
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3274

def stop_crawler(params = {}, options = {})
  req = build_request(:stop_crawler, params)
  req.send_request(options)
end

#stop_crawler_schedule(params = {}) ⇒ Struct

Sets the schedule state of the specified crawler to NOT_SCHEDULED, but does not stop the crawler if it is already running.

Examples:

Request syntax with placeholder values


resp = client.stop_crawler_schedule({
  crawler_name: "NameString", # required
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :crawler_name (required, String)

    Name of the crawler whose schedule state to set.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3297
3298
3299
3300
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3297

def stop_crawler_schedule(params = {}, options = {})
  req = build_request(:stop_crawler_schedule, params)
  req.send_request(options)
end

#stop_trigger(params = {}) ⇒ Types::StopTriggerResponse

Stops a specified trigger.

Examples:

Request syntax with placeholder values


resp = client.stop_trigger({
  name: "NameString", # required
})

Response structure


resp.name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name of the trigger to stop.

Returns:

See Also:



3325
3326
3327
3328
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3325

def stop_trigger(params = {}, options = {})
  req = build_request(:stop_trigger, params)
  req.send_request(options)
end

#update_classifier(params = {}) ⇒ Struct

Modifies an existing classifier (either a GrokClassifier or an XMLClassifier).

Examples:

Request syntax with placeholder values


resp = client.update_classifier({
  grok_classifier: {
    name: "NameString", # required
    classification: "Classification",
    grok_pattern: "GrokPattern",
    custom_patterns: "CustomPatterns",
  },
  xml_classifier: {
    name: "NameString", # required
    classification: "Classification",
    row_tag: "RowTag",
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3361
3362
3363
3364
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3361

def update_classifier(params = {}, options = {})
  req = build_request(:update_classifier, params)
  req.send_request(options)
end

#update_connection(params = {}) ⇒ Struct

Updates a connection definition in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.update_connection({
  catalog_id: "CatalogIdString",
  name: "NameString", # required
  connection_input: { # required
    name: "NameString",
    description: "DescriptionString",
    connection_type: "JDBC", # accepts JDBC, SFTP
    match_criteria: ["NameString"],
    connection_properties: {
      "HOST" => "ValueString",
    },
    physical_connection_requirements: {
      subnet_id: "NameString",
      security_group_id_list: ["NameString"],
      availability_zone: "NameString",
    },
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the connection resides. If none is supplied, the AWS account ID is used by default.

  • :name (required, String)

    The name of the connection definition to update.

  • :connection_input (required, Types::ConnectionInput)

    A ConnectionInput object that redefines the connection in question.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3405
3406
3407
3408
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3405

def update_connection(params = {}, options = {})
  req = build_request(:update_connection, params)
  req.send_request(options)
end

#update_crawler(params = {}) ⇒ Struct

Updates a crawler. If a crawler is running, you must stop it using StopCrawler before updating it.

Examples:

Request syntax with placeholder values


resp = client.update_crawler({
  name: "NameString", # required
  role: "Role",
  database_name: "DatabaseName",
  description: "DescriptionStringRemovable",
  targets: {
    s3_targets: [
      {
        path: "Path",
        exclusions: ["Path"],
      },
    ],
    jdbc_targets: [
      {
        connection_name: "ConnectionName",
        path: "Path",
        exclusions: ["Path"],
      },
    ],
  },
  schedule: "CronExpression",
  classifiers: ["NameString"],
  table_prefix: "TablePrefix",
  schema_change_policy: {
    update_behavior: "LOG", # accepts LOG, UPDATE_IN_DATABASE
    delete_behavior: "LOG", # accepts LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
  },
  configuration: "CrawlerConfiguration",
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    Name of the new crawler.

  • :role (String)

    The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

  • :database_name (String)

    The AWS Glue database where results are stored, such as: arn:aws:daylight:us-east-1::database/sometable/*.

  • :description (String)

    A description of the new crawler.

  • :targets (Types::CrawlerTargets)

    A list of targets to crawl.

  • :schedule (String)

    A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

  • :classifiers (Array<String>)

    A list of custom classifiers that the user has registered. By default, all classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

  • :table_prefix (String)

    The table prefix used for catalog tables that are created.

  • :schema_change_policy (Types::SchemaChangePolicy)

    Policy for the crawler's update and deletion behavior.

  • :configuration (String)

    Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

    You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition. Use the following JSON string to specify that behavior:

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3498
3499
3500
3501
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3498

def update_crawler(params = {}, options = {})
  req = build_request(:update_crawler, params)
  req.send_request(options)
end

#update_crawler_schedule(params = {}) ⇒ Struct

Updates the schedule of a crawler using a cron expression.

Examples:

Request syntax with placeholder values


resp = client.update_crawler_schedule({
  crawler_name: "NameString", # required
  schedule: "CronExpression",
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :crawler_name (required, String)

    Name of the crawler whose schedule to update.

  • :schedule (String)

    The updated cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3531
3532
3533
3534
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3531

def update_crawler_schedule(params = {}, options = {})
  req = build_request(:update_crawler_schedule, params)
  req.send_request(options)
end

#update_database(params = {}) ⇒ Struct

Updates an existing database definition in a Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.update_database({
  catalog_id: "CatalogIdString",
  name: "NameString", # required
  database_input: { # required
    name: "NameString", # required
    description: "DescriptionString",
    location_uri: "URI",
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog in which the metadata database resides. If none is supplied, the AWS account ID is used by default.

  • :name (required, String)

    The name of the metadata database to update in the catalog.

  • :database_input (required, Types::DatabaseInput)

    A DatabaseInput object specifying the new definition of the metadata database in the catalog.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3570
3571
3572
3573
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3570

def update_database(params = {}, options = {})
  req = build_request(:update_database, params)
  req.send_request(options)
end

#update_dev_endpoint(params = {}) ⇒ Struct

Updates a specified DevEndpoint.

Examples:

Request syntax with placeholder values


resp = client.update_dev_endpoint({
  endpoint_name: "GenericString", # required
  public_key: "GenericString",
  custom_libraries: {
    extra_python_libs_s3_path: "GenericString",
    extra_jars_s3_path: "GenericString",
  },
  update_etl_libraries: false,
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :endpoint_name (required, String)

    The name of the DevEndpoint to be updated.

  • :public_key (String)

    The public key for the DevEndpoint to use.

  • :custom_libraries (Types::DevEndpointCustomLibraries)

    Custom Python or Java libraries to be loaded in the DevEndpoint.

  • :update_etl_libraries (Boolean)

    True if the list of custom libraries to be loaded in the development endpoint needs to be updated, or False otherwise.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3608
3609
3610
3611
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3608

def update_dev_endpoint(params = {}, options = {})
  req = build_request(:update_dev_endpoint, params)
  req.send_request(options)
end

#update_job(params = {}) ⇒ Types::UpdateJobResponse

Updates an existing job definition.

Examples:

Request syntax with placeholder values


resp = client.update_job({
  job_name: "NameString", # required
  job_update: { # required
    description: "DescriptionString",
    log_uri: "UriString",
    role: "RoleString",
    execution_property: {
      max_concurrent_runs: 1,
    },
    command: {
      name: "GenericString",
      script_location: "ScriptLocationString",
    },
    default_arguments: {
      "GenericString" => "GenericString",
    },
    connections: {
      connections: ["GenericString"],
    },
    max_retries: 1,
    allocated_capacity: 1,
  },
})

Response structure


resp.job_name #=> String

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :job_name (required, String)

    Name of the job definition to update.

  • :job_update (required, Types::JobUpdate)

    Specifies the values with which to update the job.

Returns:

See Also:



3659
3660
3661
3662
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3659

def update_job(params = {}, options = {})
  req = build_request(:update_job, params)
  req.send_request(options)
end

#update_partition(params = {}) ⇒ Struct

Updates a partition.

Examples:

Request syntax with placeholder values


resp = client.update_partition({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_name: "NameString", # required
  partition_value_list: ["ValueString"], # required
  partition_input: { # required
    values: ["ValueString"],
    last_access_time: Time.now,
    storage_descriptor: {
      columns: [
        {
          name: "NameString", # required
          type: "ColumnTypeString",
          comment: "CommentString",
        },
      ],
      location: "LocationString",
      input_format: "FormatString",
      output_format: "FormatString",
      compressed: false,
      number_of_buckets: 1,
      serde_info: {
        name: "NameString",
        serialization_library: "NameString",
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
      },
      bucket_columns: ["NameString"],
      sort_columns: [
        {
          column: "NameString", # required
          sort_order: 1, # required
        },
      ],
      parameters: {
        "KeyString" => "ParametersMapValue",
      },
      skewed_info: {
        skewed_column_names: ["NameString"],
        skewed_column_values: ["ColumnValuesString"],
        skewed_column_value_location_maps: {
          "ColumnValuesString" => "ColumnValuesString",
        },
      },
      stored_as_sub_directories: false,
    },
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
    last_analyzed_time: Time.now,
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the partition to be updated resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database in which the table in question resides.

  • :table_name (required, String)

    The name of the table where the partition to be updated is located.

  • :partition_value_list (required, Array<String>)

    A list of the values defining the partition.

  • :partition_input (required, Types::PartitionInput)

    The new partition object to which to update the partition.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3745
3746
3747
3748
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3745

def update_partition(params = {}, options = {})
  req = build_request(:update_partition, params)
  req.send_request(options)
end

#update_table(params = {}) ⇒ Struct

Updates a metadata table in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.update_table({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  table_input: { # required
    name: "NameString", # required
    description: "DescriptionString",
    owner: "NameString",
    last_access_time: Time.now,
    last_analyzed_time: Time.now,
    retention: 1,
    storage_descriptor: {
      columns: [
        {
          name: "NameString", # required
          type: "ColumnTypeString",
          comment: "CommentString",
        },
      ],
      location: "LocationString",
      input_format: "FormatString",
      output_format: "FormatString",
      compressed: false,
      number_of_buckets: 1,
      serde_info: {
        name: "NameString",
        serialization_library: "NameString",
        parameters: {
          "KeyString" => "ParametersMapValue",
        },
      },
      bucket_columns: ["NameString"],
      sort_columns: [
        {
          column: "NameString", # required
          sort_order: 1, # required
        },
      ],
      parameters: {
        "KeyString" => "ParametersMapValue",
      },
      skewed_info: {
        skewed_column_names: ["NameString"],
        skewed_column_values: ["ColumnValuesString"],
        skewed_column_value_location_maps: {
          "ColumnValuesString" => "ColumnValuesString",
        },
      },
      stored_as_sub_directories: false,
    },
    partition_keys: [
      {
        name: "NameString", # required
        type: "ColumnTypeString",
        comment: "CommentString",
      },
    ],
    view_original_text: "ViewTextString",
    view_expanded_text: "ViewTextString",
    table_type: "TableTypeString",
    parameters: {
      "KeyString" => "ParametersMapValue",
    },
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the table resides. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database in which the table resides.

  • :table_input (required, Types::TableInput)

    An updated TableInput object to define the metadata table in the catalog.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3836
3837
3838
3839
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3836

def update_table(params = {}, options = {})
  req = build_request(:update_table, params)
  req.send_request(options)
end

#update_trigger(params = {}) ⇒ Types::UpdateTriggerResponse

Updates a trigger definition.

Examples:

Request syntax with placeholder values


resp = client.update_trigger({
  name: "NameString", # required
  trigger_update: { # required
    name: "NameString",
    description: "DescriptionString",
    schedule: "GenericString",
    actions: [
      {
        job_name: "NameString",
        arguments: {
          "GenericString" => "GenericString",
        },
      },
    ],
    predicate: {
      logical: "AND", # accepts AND
      conditions: [
        {
          logical_operator: "EQUALS", # accepts EQUALS
          job_name: "NameString",
          state: "STARTING", # accepts STARTING, RUNNING, STOPPING, STOPPED, SUCCEEDED, FAILED
        },
      ],
    },
  },
})

Response structure


resp.trigger.name #=> String
resp.trigger.id #=> String
resp.trigger.type #=> String, one of "SCHEDULED", "CONDITIONAL", "ON_DEMAND"
resp.trigger.state #=> String, one of "CREATING", "CREATED", "ACTIVATING", "ACTIVATED", "DEACTIVATING", "DEACTIVATED", "DELETING", "UPDATING"
resp.trigger.description #=> String
resp.trigger.schedule #=> String
resp.trigger.actions #=> Array
resp.trigger.actions[0].job_name #=> String
resp.trigger.actions[0].arguments #=> Hash
resp.trigger.actions[0].arguments["GenericString"] #=> String
resp.trigger.predicate.logical #=> String, one of "AND"
resp.trigger.predicate.conditions #=> Array
resp.trigger.predicate.conditions[0].logical_operator #=> String, one of "EQUALS"
resp.trigger.predicate.conditions[0].job_name #=> String
resp.trigger.predicate.conditions[0].state #=> String, one of "STARTING", "RUNNING", "STOPPING", "STOPPED", "SUCCEEDED", "FAILED"

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :name (required, String)

    The name of the trigger to update.

  • :trigger_update (required, Types::TriggerUpdate)

    The new values with which to update the trigger.

Returns:

See Also:



3904
3905
3906
3907
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3904

def update_trigger(params = {}, options = {})
  req = build_request(:update_trigger, params)
  req.send_request(options)
end

#update_user_defined_function(params = {}) ⇒ Struct

Updates an existing function definition in the Data Catalog.

Examples:

Request syntax with placeholder values


resp = client.update_user_defined_function({
  catalog_id: "CatalogIdString",
  database_name: "NameString", # required
  function_name: "NameString", # required
  function_input: { # required
    function_name: "NameString",
    class_name: "NameString",
    owner_name: "NameString",
    owner_type: "USER", # accepts USER, ROLE, GROUP
    resource_uris: [
      {
        resource_type: "JAR", # accepts JAR, FILE, ARCHIVE
        uri: "URI",
      },
    ],
  },
})

Parameters:

  • params (Hash) (defaults to: {})

    ({})

Options Hash (params):

  • :catalog_id (String)

    The ID of the Data Catalog where the function to be updated is located. If none is supplied, the AWS account ID is used by default.

  • :database_name (required, String)

    The name of the catalog database where the function to be updated is located.

  • :function_name (required, String)

    The name of the function.

  • :function_input (required, Types::UserDefinedFunctionInput)

    A FunctionInput object that re-defines the function in the Data Catalog.

Returns:

  • (Struct)

    Returns an empty response.

See Also:



3952
3953
3954
3955
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/client.rb', line 3952

def update_user_defined_function(params = {}, options = {})
  req = build_request(:update_user_defined_function, params)
  req.send_request(options)
end