CfnCrawlerProps

class aws_cdk.aws_glue.CfnCrawlerProps(*, role, targets, classifiers=None, configuration=None, crawler_security_configuration=None, database_name=None, description=None, name=None, recrawl_policy=None, schedule=None, schema_change_policy=None, table_prefix=None, tags=None)

Bases: object

Properties for defining a CfnCrawler.

Parameters
  • role (str) – The Amazon Resource Name (ARN) of an IAM role that’s used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

  • targets (Union[IResolvable, TargetsProperty, Dict[str, Any]]) – A collection of targets to crawl.

  • classifiers (Optional[Sequence[str]]) – A list of UTF-8 strings that specify the names of custom classifiers that are associated with the crawler.

  • configuration (Optional[str]) – Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler’s behavior. For more information, see Configuring a Crawler .

  • crawler_security_configuration (Optional[str]) – The name of the SecurityConfiguration structure to be used by this crawler.

  • database_name (Optional[str]) – The name of the database in which the crawler’s output is stored.

  • description (Optional[str]) – A description of the crawler.

  • name (Optional[str]) – The name of the crawler.

  • recrawl_policy (Union[IResolvable, RecrawlPolicyProperty, Dict[str, Any], None]) – A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

  • schedule (Union[IResolvable, ScheduleProperty, Dict[str, Any], None]) – For scheduled crawlers, the schedule when the crawler runs.

  • schema_change_policy (Union[IResolvable, SchemaChangePolicyProperty, Dict[str, Any], None]) – The policy that specifies update and delete behaviors for the crawler. The policy tells the crawler what to do in the event that it detects a change in a table that already exists in the customer’s database at the time of the crawl. The SchemaChangePolicy does not affect whether or how new tables and partitions are added. New tables and partitions are always created regardless of the SchemaChangePolicy on a crawler. The SchemaChangePolicy consists of two components, UpdateBehavior and DeleteBehavior .

  • table_prefix (Optional[str]) – The prefix added to the names of tables that are created.

  • tags (Optional[Any]) – The tags to use with this crawler.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html

ExampleMetadata

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_glue as glue

# tags: Any

cfn_crawler_props = glue.CfnCrawlerProps(
    role="role",
    targets=glue.CfnCrawler.TargetsProperty(
        catalog_targets=[glue.CfnCrawler.CatalogTargetProperty(
            database_name="databaseName",
            tables=["tables"]
        )],
        dynamo_db_targets=[glue.CfnCrawler.DynamoDBTargetProperty(
            path="path"
        )],
        jdbc_targets=[glue.CfnCrawler.JdbcTargetProperty(
            connection_name="connectionName",
            exclusions=["exclusions"],
            path="path"
        )],
        mongo_db_targets=[glue.CfnCrawler.MongoDBTargetProperty(
            connection_name="connectionName",
            path="path"
        )],
        s3_targets=[glue.CfnCrawler.S3TargetProperty(
            connection_name="connectionName",
            dlq_event_queue_arn="dlqEventQueueArn",
            event_queue_arn="eventQueueArn",
            exclusions=["exclusions"],
            path="path",
            sample_size=123
        )]
    ),

    # the properties below are optional
    classifiers=["classifiers"],
    configuration="configuration",
    crawler_security_configuration="crawlerSecurityConfiguration",
    database_name="databaseName",
    description="description",
    name="name",
    recrawl_policy=glue.CfnCrawler.RecrawlPolicyProperty(
        recrawl_behavior="recrawlBehavior"
    ),
    schedule=glue.CfnCrawler.ScheduleProperty(
        schedule_expression="scheduleExpression"
    ),
    schema_change_policy=glue.CfnCrawler.SchemaChangePolicyProperty(
        delete_behavior="deleteBehavior",
        update_behavior="updateBehavior"
    ),
    table_prefix="tablePrefix",
    tags=tags
)

Attributes

classifiers

A list of UTF-8 strings that specify the names of custom classifiers that are associated with the crawler.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-classifiers

Return type

Optional[List[str]]

configuration

Crawler configuration information.

This versioned JSON string allows users to specify aspects of a crawler’s behavior. For more information, see Configuring a Crawler .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-configuration

Return type

Optional[str]

crawler_security_configuration

The name of the SecurityConfiguration structure to be used by this crawler.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-crawlersecurityconfiguration

Return type

Optional[str]

database_name

The name of the database in which the crawler’s output is stored.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-databasename

Return type

Optional[str]

description

A description of the crawler.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-description

Return type

Optional[str]

name

The name of the crawler.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-name

Return type

Optional[str]

recrawl_policy

A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-recrawlpolicy

Return type

Union[IResolvable, RecrawlPolicyProperty, None]

role

The Amazon Resource Name (ARN) of an IAM role that’s used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-role

Return type

str

schedule

For scheduled crawlers, the schedule when the crawler runs.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-schedule

Return type

Union[IResolvable, ScheduleProperty, None]

schema_change_policy

The policy that specifies update and delete behaviors for the crawler.

The policy tells the crawler what to do in the event that it detects a change in a table that already exists in the customer’s database at the time of the crawl. The SchemaChangePolicy does not affect whether or how new tables and partitions are added. New tables and partitions are always created regardless of the SchemaChangePolicy on a crawler.

The SchemaChangePolicy consists of two components, UpdateBehavior and DeleteBehavior .

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-schemachangepolicy

Return type

Union[IResolvable, SchemaChangePolicyProperty, None]

table_prefix

The prefix added to the names of tables that are created.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-tableprefix

Return type

Optional[str]

tags

The tags to use with this crawler.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-tags

Return type

Any

targets

A collection of targets to crawl.

Link

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-crawler.html#cfn-glue-crawler-targets

Return type

Union[IResolvable, TargetsProperty]