You are viewing documentation for version 3 of the AWS SDK for Ruby. Version 2 documentation can be found here.

Class: Aws::Glue::Types::CreateCrawlerRequest

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Note:

When making an API call, you may pass CreateCrawlerRequest data as a hash:

{
  name: "NameString", # required
  role: "Role", # required
  database_name: "DatabaseName",
  description: "DescriptionString",
  targets: { # required
    s3_targets: [
      {
        path: "Path",
        exclusions: ["Path"],
      },
    ],
    jdbc_targets: [
      {
        connection_name: "ConnectionName",
        path: "Path",
        exclusions: ["Path"],
      },
    ],
    dynamo_db_targets: [
      {
        path: "Path",
      },
    ],
    catalog_targets: [
      {
        database_name: "NameString", # required
        tables: ["NameString"], # required
      },
    ],
  },
  schedule: "CronExpression",
  classifiers: ["NameString"],
  table_prefix: "TablePrefix",
  schema_change_policy: {
    update_behavior: "LOG", # accepts LOG, UPDATE_IN_DATABASE
    delete_behavior: "LOG", # accepts LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
  },
  configuration: "CrawlerConfiguration",
  crawler_security_configuration: "CrawlerSecurityConfiguration",
  tags: {
    "TagKey" => "TagValue",
  },
}

Instance Attribute Summary collapse

Instance Attribute Details

#classifiersArray<String>

A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

Returns:

  • (Array<String>)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#configurationString

The crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#crawler_security_configurationString

The name of the SecurityConfiguration structure to be used by this crawler.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#database_nameString

The AWS Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/*.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#descriptionString

A description of the new crawler.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#nameString

Name of the new crawler.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#roleString

The IAM role or Amazon Resource Name (ARN) of an IAM role used by the new crawler to access customer resources.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#scheduleString

A cron expression used to specify the schedule. For more information, see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, specify cron(15 12 * * ? *).

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#schema_change_policyTypes::SchemaChangePolicy

The policy for the crawler's update and deletion behavior.



1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#table_prefixString

The table prefix used for catalog tables that are created.

Returns:

  • (String)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#tagsHash<String,String>

The tags to use with this crawler request. You can use tags to limit access to the crawler. For more information, see AWS Tags in AWS Glue.

Returns:

  • (Hash<String,String>)


1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end

#targetsTypes::CrawlerTargets

A list of collection of targets to crawl.



1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1949

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration,
  :crawler_security_configuration,
  :tags)
  include Aws::Structure
end