You are viewing documentation for version 3 of the AWS SDK for Ruby. Version 2 documentation can be found here.

Class: Aws::Glue::Types::CreateCrawlerRequest

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Note:

When making an API call, you may pass CreateCrawlerRequest data as a hash:

{
  name: "NameString", # required
  role: "Role", # required
  database_name: "DatabaseName", # required
  description: "DescriptionString",
  targets: { # required
    s3_targets: [
      {
        path: "Path",
        exclusions: ["Path"],
      },
    ],
    jdbc_targets: [
      {
        connection_name: "ConnectionName",
        path: "Path",
        exclusions: ["Path"],
      },
    ],
  },
  schedule: "CronExpression",
  classifiers: ["NameString"],
  table_prefix: "TablePrefix",
  schema_change_policy: {
    update_behavior: "LOG", # accepts LOG, UPDATE_IN_DATABASE
    delete_behavior: "LOG", # accepts LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
  },
  configuration: "CrawlerConfiguration",
}

Instance Attribute Summary collapse

Instance Attribute Details

#classifiersArray<String>

A list of custom classifiers that the user has registered. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.

Returns:

  • (Array<String>)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#configurationString

Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition.

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#database_nameString

The AWS Glue database where results are written, such as: arn:aws:daylight:us-east-1::database/sometable/*.

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#descriptionString

A description of the new crawler.

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#nameString

Name of the new crawler.

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#roleString

The IAM role (or ARN of an IAM role) used by the new crawler to access customer resources.

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#scheduleString

A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#schema_change_policyTypes::SchemaChangePolicy

Policy for the crawler's update and deletion behavior.



1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#table_prefixString

The table prefix used for catalog tables that are created.

Returns:

  • (String)


1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end

#targetsTypes::CrawlerTargets

A list of collection of targets to crawl.



1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 1161

class CreateCrawlerRequest < Struct.new(
  :name,
  :role,
  :database_name,
  :description,
  :targets,
  :schedule,
  :classifiers,
  :table_prefix,
  :schema_change_policy,
  :configuration)
  include Aws::Structure
end