You are viewing documentation for version 3 of the AWS SDK for Ruby. Version 2 documentation can be found here.

Class: Aws::Glue::Types::Crawler

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.

Instance Attribute Summary collapse

Instance Attribute Details

#classifiersArray<String>

A list of custom classifiers associated with the crawler.

Returns:

  • (Array<String>)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#configurationString

Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition. Use the following JSON string to specify that behavior:

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#crawl_elapsed_timeInteger

If the crawler is running, contains the total time elapsed since the last crawl began.

Returns:

  • (Integer)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#creation_timeTime

The time when the crawler was created.

Returns:

  • (Time)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#database_nameString

The database where metadata is written by this crawler.

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#descriptionString

A description of the crawler.

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#last_crawlTypes::LastCrawlInfo

The status of the last crawl, and potentially error information if an error occurred.



884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#last_updatedTime

The time the crawler was last updated.

Returns:

  • (Time)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#nameString

The crawler name.

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#roleString

The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#scheduleTypes::Schedule

For scheduled crawlers, the schedule when the crawler runs.

Returns:



884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#schema_change_policyTypes::SchemaChangePolicy

Sets the behavior when the crawler finds a changed or deleted object.



884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#stateString

Indicates whether the crawler is running, or whether a run is pending.

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#table_prefixString

The prefix added to the names of tables that are created.

Returns:

  • (String)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#targetsTypes::CrawlerTargets

A collection of targets to crawl.



884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#versionInteger

The version of the crawler.

Returns:

  • (Integer)


884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 884

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end