You are viewing documentation for version 3 of the AWS SDK for Ruby. Version 2 documentation can be found here.

Class: Aws::Glue::Types::Crawler

Inherits:
Struct
  • Object
show all
Defined in:
gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb

Overview

Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the AWS Glue Data Catalog.

Instance Attribute Summary collapse

Instance Attribute Details

#classifiersArray<String>

A list of custom classifiers associated with the crawler.

Returns:

  • (Array<String>)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#configurationString

Crawler configuration information. This versioned JSON string allows users to specify aspects of a Crawler's behavior.

You can use this field to force partitions to inherit metadata such as classification, input format, output format, serde information, and schema from their parent table, rather than detect this information separately for each partition. Use the following JSON string to specify that behavior:

Example: '\{ "Version": 1.0, "CrawlerOutput": \{ "Partitions": \{ "AddOrUpdateBehavior": "InheritFromTable" \} \} \}'

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#crawl_elapsed_timeInteger

If the crawler is running, contains the total time elapsed since the last crawl began.

Returns:

  • (Integer)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#creation_timeTime

The time when the crawler was created.

Returns:

  • (Time)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#database_nameString

The database where metadata is written by this crawler.

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#descriptionString

A description of the crawler.

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#last_crawlTypes::LastCrawlInfo

The status of the last crawl, and potentially error information if an error occurred.



957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#last_updatedTime

The time the crawler was last updated.

Returns:

  • (Time)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#nameString

The crawler name.

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#roleString

The IAM role (or ARN of an IAM role) used to access customer resources, such as data in Amazon S3.

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#scheduleTypes::Schedule

For scheduled crawlers, the schedule when the crawler runs.

Returns:



957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#schema_change_policyTypes::SchemaChangePolicy

Sets the behavior when the crawler finds a changed or deleted object.



957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#stateString

Indicates whether the crawler is running, or whether a run is pending.

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#table_prefixString

The prefix added to the names of tables that are created.

Returns:

  • (String)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#targetsTypes::CrawlerTargets

A collection of targets to crawl.



957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end

#versionInteger

The version of the crawler.

Returns:

  • (Integer)


957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
# File 'gems/aws-sdk-glue/lib/aws-sdk-glue/types.rb', line 957

class Crawler < Struct.new(
  :name,
  :role,
  :targets,
  :database_name,
  :description,
  :classifiers,
  :schema_change_policy,
  :state,
  :table_prefix,
  :schedule,
  :crawl_elapsed_time,
  :creation_time,
  :last_updated,
  :last_crawl,
  :version,
  :configuration)
  include Aws::Structure
end