You are viewing documentation for version 3 of the AWS SDK for Ruby. Version 2 documentation can be found here.

Class: Aws::Glue::Types::Crawler

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.

See Also:

Instance Attribute Summary collapse

Instance Attribute Details

#classifiers ⇒ Array<String>

A list of custom classifiers associated with the crawler.

Returns:

  • (Array<String>) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#configuration ⇒ String

Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition. Use the following JSON string to specify that behavior:

Example: '\{ "Version": 1.0, "CrawlerOutput": \{ "Partitions": \{ "AddOrUpdateBehavior": "InheritFromTable" \} \} \}'

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#crawl_elapsed_time ⇒ Integer

If the crawler is running, contains the total time elapsed since the last crawl began.

Returns:

  • (Integer) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#creation_time ⇒ Time

The time when the crawler was created.

Returns:

  • (Time) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#database_name ⇒ String

The database where metadata is written by this crawler.

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#description ⇒ String

A description of the crawler.

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#last_crawl ⇒ Types::LastCrawlInfo

The status of the last crawl, and potentially error information if an error occurred.

Returns:



979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#last_updated ⇒ Time

The time the crawler was last updated.

Returns:

  • (Time) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#name ⇒ String

The crawler name.

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#role ⇒ String

The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#schedule ⇒ Types::Schedule

For scheduled crawlers, the schedule when the crawler runs.

Returns:



979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#schema_change_policy ⇒ Types::SchemaChangePolicy

Sets the behavior when the crawler finds a changed or deleted object.

Returns:



979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#state ⇒ String

Indicates whether the crawler is running, or whether a run is pending.

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#table_prefix ⇒ String

The prefix added to the names of tables that are created.

Returns:

  • (String) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#targets ⇒ Types::CrawlerTargets

A collection of targets to crawl.

Returns:



979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#version ⇒ Integer

The version of the crawler.

Returns:

  • (Integer) 


979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
 
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 979

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end