Class: Aws::Glue::Types::Crawler
- Inherits:
-
Struct
- Object
- Struct
- Aws::Glue::Types::Crawler
- Defined in:
- gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb
Overview
Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the Glue Data Catalog.
Constant Summary collapse
- SENSITIVE =
[]
Instance Attribute Summary collapse
-
#classifiers ⇒ Array<String>
A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.
-
#configuration ⇒ String
Crawler configuration information.
-
#crawl_elapsed_time ⇒ Integer
If the crawler is running, contains the total time elapsed since the last crawl began.
-
#crawler_security_configuration ⇒ String
The name of the
SecurityConfiguration
structure to be used by this crawler. -
#creation_time ⇒ Time
The time that the crawler was created.
-
#database_name ⇒ String
The name of the database in which the crawler's output is stored.
-
#description ⇒ String
A description of the crawler.
-
#lake_formation_configuration ⇒ Types::LakeFormationConfiguration
Specifies whether the crawler should use AWS Lake Formation credentials for the crawler instead of the IAM role credentials.
-
#last_crawl ⇒ Types::LastCrawlInfo
The status of the last crawl, and potentially error information if an error occurred.
-
#last_updated ⇒ Time
The time that the crawler was last updated.
-
#lineage_configuration ⇒ Types::LineageConfiguration
A configuration that specifies whether data lineage is enabled for the crawler.
-
#name ⇒ String
The name of the crawler.
-
#recrawl_policy ⇒ Types::RecrawlPolicy
A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.
-
#role ⇒ String
The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.
-
#schedule ⇒ Types::Schedule
For scheduled crawlers, the schedule when the crawler runs.
-
#schema_change_policy ⇒ Types::SchemaChangePolicy
The policy that specifies update and delete behaviors for the crawler.
-
#state ⇒ String
Indicates whether the crawler is running, or whether a run is pending.
-
#table_prefix ⇒ String
The prefix added to the names of tables that are created.
-
#targets ⇒ Types::CrawlerTargets
A collection of targets to crawl.
-
#version ⇒ Integer
The version of the crawler.
Instance Attribute Details
#classifiers ⇒ Array<String>
A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#configuration ⇒ String
Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Include and Exclude Patterns.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#crawl_elapsed_time ⇒ Integer
If the crawler is running, contains the total time elapsed since the last crawl began.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#crawler_security_configuration ⇒ String
The name of the SecurityConfiguration
structure to be used by this
crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#creation_time ⇒ Time
The time that the crawler was created.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#database_name ⇒ String
The name of the database in which the crawler's output is stored.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#description ⇒ String
A description of the crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#lake_formation_configuration ⇒ Types::LakeFormationConfiguration
Specifies whether the crawler should use AWS Lake Formation credentials for the crawler instead of the IAM role credentials.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#last_crawl ⇒ Types::LastCrawlInfo
The status of the last crawl, and potentially error information if an error occurred.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#last_updated ⇒ Time
The time that the crawler was last updated.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#lineage_configuration ⇒ Types::LineageConfiguration
A configuration that specifies whether data lineage is enabled for the crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#name ⇒ String
The name of the crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#recrawl_policy ⇒ Types::RecrawlPolicy
A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#role ⇒ String
The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#schedule ⇒ Types::Schedule
For scheduled crawlers, the schedule when the crawler runs.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#schema_change_policy ⇒ Types::SchemaChangePolicy
The policy that specifies update and delete behaviors for the crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#state ⇒ String
Indicates whether the crawler is running, or whether a run is pending.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#table_prefix ⇒ String
The prefix added to the names of tables that are created.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#targets ⇒ Types::CrawlerTargets
A collection of targets to crawl.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |
#version ⇒ Integer
The version of the crawler.
4133 4134 4135 4136 4137 4138 4139 4140 4141 4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 |
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 4133 class Crawler < Struct.new( :name, :role, :targets, :database_name, :description, :classifiers, :recrawl_policy, :schema_change_policy, :lineage_configuration, :state, :table_prefix, :schedule, :crawl_elapsed_time, :creation_time, :last_updated, :last_crawl, :version, :configuration, :crawler_security_configuration, :lake_formation_configuration) SENSITIVE = [] include Aws::Structure end |