AWS Glue
Developer Guide

Classifier API

The Classifier API describes AWS Glue classifier data types, and includes the API for creating, deleting, updating, and listing classifiers.

Data Types

Classifier Structure

Classifiers are triggered during a crawl task. A classifier checks whether a given file is in a format it can handle, and if it is, the classifier creates a schema in the form of a StructType object that matches that data format.

You can use the standard classifiers that AWS Glue supplies, or you can write your own classifiers to best categorize your data sources and specify the appropriate schemas to use for them. A classifier can be a grok classifier, an XML classifier, or a JSON classifier, as specified in one of the fields in the Classifier object.

Fields

  • GrokClassifier – A GrokClassifier object.

    A GrokClassifier object.

  • XMLClassifier – A XMLClassifier object.

    An XMLClassifier object.

  • JsonClassifier – A JsonClassifier object.

    A JsonClassifier object.

GrokClassifier Structure

A classifier that uses grok patterns.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • ClassificationRequired: UTF-8 string.

    An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, and so on.

  • CreationTime – Timestamp.

    The time this classifier was registered.

  • LastUpdated – Timestamp.

    The time this classifier was last updated.

  • Version – Number (long).

    The version of this classifier.

  • GrokPatternRequired: UTF-8 string, not less than 1 or more than 2048 bytes long, matching the A Logstash Grok string pattern.

    The grok pattern applied to a data store by this classifier. For more information, see built-in patterns in Writing Custom Classifers.

  • CustomPatterns – UTF-8 string, not more than 16000 bytes long, matching the URI address multi-line string pattern.

    Optional custom grok patterns defined by this classifier. For more information, see custom patterns in Writing Custom Classifers.

XMLClassifier Structure

A classifier for XML content.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • ClassificationRequired: UTF-8 string.

    An identifier of the data format that the classifier matches.

  • CreationTime – Timestamp.

    The time this classifier was registered.

  • LastUpdated – Timestamp.

    The time this classifier was last updated.

  • Version – Number (long).

    The version of this classifier.

  • RowTag – UTF-8 string.

    The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by />). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).

JsonClassifier Structure

A classifier for JSON content.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • CreationTime – Timestamp.

    The time this classifier was registered.

  • LastUpdated – Timestamp.

    The time this classifier was last updated.

  • Version – Number (long).

    The version of this classifier.

  • JsonPathRequired: UTF-8 string.

    A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.

CreateGrokClassifierRequest Structure

Specifies a grok classifier for CreateClassifier to create.

Fields

  • ClassificationRequired: UTF-8 string.

    An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the new classifier.

  • GrokPatternRequired: UTF-8 string, not less than 1 or more than 2048 bytes long, matching the A Logstash Grok string pattern.

    The grok pattern used by this classifier.

  • CustomPatterns – UTF-8 string, not more than 16000 bytes long, matching the URI address multi-line string pattern.

    Optional custom grok patterns used by this classifier.

UpdateGrokClassifierRequest Structure

Specifies a grok classifier to update when passed to UpdateClassifier.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the GrokClassifier.

  • Classification – UTF-8 string.

    An identifier of the data format that the classifier matches, such as Twitter, JSON, Omniture logs, Amazon CloudWatch Logs, and so on.

  • GrokPattern – UTF-8 string, not less than 1 or more than 2048 bytes long, matching the A Logstash Grok string pattern.

    The grok pattern used by this classifier.

  • CustomPatterns – UTF-8 string, not more than 16000 bytes long, matching the URI address multi-line string pattern.

    Optional custom grok patterns used by this classifier.

CreateXMLClassifierRequest Structure

Specifies an XML classifier for CreateClassifier to create.

Fields

  • ClassificationRequired: UTF-8 string.

    An identifier of the data format that the classifier matches.

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • RowTag – UTF-8 string.

    The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by />). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).

UpdateXMLClassifierRequest Structure

Specifies an XML classifier to be updated.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • Classification – UTF-8 string.

    An identifier of the data format that the classifier matches.

  • RowTag – UTF-8 string.

    The XML tag designating the element that contains each record in an XML document being parsed. Note that this cannot identify a self-closing element (closed by />). An empty row element that contains only attributes can be parsed as long as it ends with a closing tag (for example, <row item_a="A" item_b="B"></row> is okay, but <row item_a="A" item_b="B" /> is not).

CreateJsonClassifierRequest Structure

Specifies a JSON classifier for CreateClassifier to create.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • JsonPathRequired: UTF-8 string.

    A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.

UpdateJsonClassifierRequest Structure

Specifies a JSON classifier to be updated.

Fields

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The name of the classifier.

  • JsonPath – UTF-8 string.

    A JsonPath string defining the JSON data for the classifier to classify. AWS Glue supports a subset of JsonPath, as described in Writing JsonPath Custom Classifiers.

Operations

CreateClassifier Action (Python: create_classifier)

Creates a classifier in the user's account. This may be a GrokClassifier, an XMLClassifier, or abbrev JsonClassifier, depending on which field of the request is present.

Request

Response

  • No Response parameters.

Errors

  • AlreadyExistsException

  • InvalidInputException

  • OperationTimeoutException

DeleteClassifier Action (Python: delete_classifier)

Removes a classifier from the Data Catalog.

Request

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    Name of the classifier to remove.

Response

  • No Response parameters.

Errors

  • EntityNotFoundException

  • OperationTimeoutException

GetClassifier Action (Python: get_classifier)

Retrieve a classifier by name.

Request

  • NameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    Name of the classifier to retrieve.

Response

  • Classifier – A Classifier object.

    The requested classifier.

Errors

  • EntityNotFoundException

  • OperationTimeoutException

GetClassifiers Action (Python: get_classifiers)

Lists all classifier objects in the Data Catalog.

Request

  • MaxResults – Number (integer), not less than 1 or more than 1000.

    Size of the list to return (optional).

  • NextToken – UTF-8 string.

    An optional continuation token.

Response

  • Classifiers – An array of Classifier objects.

    The requested list of classifier objects.

  • NextToken – UTF-8 string.

    A continuation token.

Errors

  • OperationTimeoutException

UpdateClassifier Action (Python: update_classifier)

Modifies an existing classifier (a GrokClassifier, XMLClassifier, or JsonClassifier, depending on which field is present).

Request

Response

  • No Response parameters.

Errors

  • InvalidInputException

  • VersionMismatchException

  • EntityNotFoundException

  • OperationTimeoutException