Class: Aws::Glue::Types::UpdateCrawlerRequest
- Inherits:
-
Struct
- Object
- Struct
- Aws::Glue::Types::UpdateCrawlerRequest
- Defined in:
- gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb
Overview
When making an API call, you may pass UpdateCrawlerRequest data as a hash:
{
name: "NameString", # required
role: "Role",
database_name: "DatabaseName",
description: "DescriptionStringRemovable",
targets: {
s3_targets: [
{
path: "Path",
exclusions: ["Path"],
connection_name: "ConnectionName",
sample_size: 1,
event_queue_arn: "EventQueueArn",
dlq_event_queue_arn: "EventQueueArn",
},
],
jdbc_targets: [
{
connection_name: "ConnectionName",
path: "Path",
exclusions: ["Path"],
},
],
mongo_db_targets: [
{
connection_name: "ConnectionName",
path: "Path",
scan_all: false,
},
],
dynamo_db_targets: [
{
path: "Path",
scan_all: false,
scan_rate: 1.0,
},
],
catalog_targets: [
{
database_name: "NameString", # required
tables: ["NameString"], # required
connection_name: "ConnectionName",
},
],
delta_targets: [
{
delta_tables: ["Path"],
connection_name: "ConnectionName",
write_manifest: false,
},
],
},
schedule: "CronExpression",
classifiers: ["NameString"],
table_prefix: "TablePrefix",
schema_change_policy: {
update_behavior: "LOG", # accepts LOG, UPDATE_IN_DATABASE
delete_behavior: "LOG", # accepts LOG, DELETE_FROM_DATABASE, DEPRECATE_IN_DATABASE
},
recrawl_policy: {
recrawl_behavior: "CRAWL_EVERYTHING", # accepts CRAWL_EVERYTHING, CRAWL_NEW_FOLDERS_ONLY, CRAWL_EVENT_MODE
},
lineage_configuration: {
crawler_lineage_settings: "ENABLE", # accepts ENABLE, DISABLE
},
lake_formation_configuration: {
use_lake_formation_credentials: false,
account_id: "AccountId",
},
configuration: "CrawlerConfiguration",
crawler_security_configuration: "CrawlerSecurityConfiguration",
}
Constant Summary collapse
- SENSITIVE =
[]
Instance Attribute Summary collapse
-
#classifiers ⇒ Array<String>
A list of custom classifiers that the user has registered.
-
#configuration ⇒ String
Crawler configuration information.
-
#crawler_security_configuration ⇒ String
The name of the
SecurityConfiguration
structure to be used by this crawler. -
#database_name ⇒ String
The Glue database where results are stored, such as:
arn:aws:daylight:us-east-1::database/sometable/*
. -
#description ⇒ String
A description of the new crawler.
-
#lake_formation_configuration ⇒ Types::LakeFormationConfiguration
Specifies Lake Formation configuration settings for the crawler.
-
#lineage_configuration ⇒ Types::LineageConfiguration
Specifies data lineage configuration settings for the crawler.
-
#name ⇒ String
Name of the new crawler.
-
#recrawl_policy ⇒ Types::RecrawlPolicy
A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.
-
#role ⇒ String
The IAM role or Amazon Resource Name (ARN) of an IAM role that is used by the new crawler to access customer resources.
-
#schedule ⇒ String
A
cron
expression used to specify the schedule (see [Time-Based Schedules for Jobs and Crawlers][1]. For example, to run something every day at 12:15 UTC, you would specify:cron(15 12 * * ? *)
. -
#schema_change_policy ⇒ Types::SchemaChangePolicy
The policy for the crawler's update and deletion behavior.
-
#table_prefix ⇒ String
The table prefix used for catalog tables that are created.
-
#targets ⇒ Types::CrawlerTargets
A list of targets to crawl.
Instance Attribute Details
#classifiers ⇒ Array<String>
A list of custom classifiers that the user has registered. By default, all built-in classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#configuration ⇒ String
Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#crawler_security_configuration ⇒ String
The name of the SecurityConfiguration
structure to be used by this
crawler.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#database_name ⇒ String
The Glue database where results are stored, such as:
arn:aws:daylight:us-east-1::database/sometable/*
.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#description ⇒ String
A description of the new crawler.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#lake_formation_configuration ⇒ Types::LakeFormationConfiguration
Specifies Lake Formation configuration settings for the crawler.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#lineage_configuration ⇒ Types::LineageConfiguration
Specifies data lineage configuration settings for the crawler.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#name ⇒ String
Name of the new crawler.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#recrawl_policy ⇒ Types::RecrawlPolicy
A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#role ⇒ String
The IAM role or Amazon Resource Name (ARN) of an IAM role that is used by the new crawler to access customer resources.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#schedule ⇒ String
A cron
expression used to specify the schedule (see Time-Based
Schedules for Jobs and Crawlers. For example, to run something
every day at 12:15 UTC, you would specify: cron(15 12 * * ? *)
.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#schema_change_policy ⇒ Types::SchemaChangePolicy
The policy for the crawler's update and deletion behavior.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#table_prefix ⇒ String
The table prefix used for catalog tables that are created.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |
#targets ⇒ Types::CrawlerTargets
A list of targets to crawl.
24434 24435 24436 24437 24438 24439 24440 24441 24442 24443 24444 24445 24446 24447 24448 24449 24450 24451 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 24434 class UpdateCrawlerRequest < Struct.new( :name, :role, :database_name, :description, :targets, :schedule, :classifiers, :table_prefix, :schema_change_policy, :recrawl_policy, :lineage_configuration, :lake_formation_configuration, :configuration, :crawler_security_configuration) SENSITIVE = [] include Aws::Structure end |