Menu
AWS Glue
Web API Reference (API Version 2017-03-31)

Crawler

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.

Contents

Classifiers

A list of custom classifiers associated with the crawler.

Type: Array of strings

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*

Required: No

Configuration

Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Configuring a Crawler.

Type: String

Required: No

CrawlElapsedTime

If the crawler is running, contains the total time elapsed since the last crawl began.

Type: Long

Required: No

CreationTime

The time when the crawler was created.

Type: Timestamp

Required: No

DatabaseName

The database where metadata is written by this crawler.

Type: String

Required: No

Description

A description of the crawler.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 2048.

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*

Required: No

LastCrawl

The status of the last crawl, and potentially error information if an error occurred.

Type: LastCrawlInfo object

Required: No

LastUpdated

The time the crawler was last updated.

Type: Timestamp

Required: No

Name

The crawler name.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: [\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*

Required: No

Role

The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.

Type: String

Required: No

Schedule

For scheduled crawlers, the schedule when the crawler runs.

Type: Schedule object

Required: No

SchemaChangePolicy

Sets the behavior when the crawler finds a changed or deleted object.

Type: SchemaChangePolicy object

Required: No

State

Indicates whether the crawler is running, or whether a run is pending.

Type: String

Valid Values: READY | RUNNING | STOPPING

Required: No

TablePrefix

The prefix added to the names of tables that are created.

Type: String

Length Constraints: Minimum length of 0. Maximum length of 128.

Required: No

Targets

A collection of targets to crawl.

Type: CrawlerTargets object

Required: No

Version

The version of the crawler.

Type: Long

Required: No

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following:

On this page: