Object APIs - AWS Lake Formation

Object APIs

The Object APIs describes governed table objects in AWS Lake Formation.

Data Types

TableObject Structure

Specifies the details of a governed table.

Fields

  • Uri – Uniform resource identifier (uri), not less than 1 or more than 1024 bytes long, matching the URI address multi-line string pattern.

    The Amazon S3 location of the object.

  • ETag – UTF-8 string, not less than 1 or more than 255 bytes long, matching the Custom string pattern #11.

    The Amazon S3 ETag of the object. Returned by GetTableObjects for validation and used to identify changes to the underlying data.

  • Size – Number (long).

    The size of the Amazon S3 object in bytes.

PartitionObjects Structure

A structure containing a list of partition values and table objects.

Fields

  • PartitionValues – An array of UTF-8 strings, not less than 1 or more than 100 strings.

    A list of partition values.

  • Objects – An array of TableObject objects.

    A list of table objects

AddObjectInput Structure

A new object to add to the governed table.

Fields

  • UriRequired: Uniform resource identifier (uri), not less than 1 or more than 1024 bytes long, matching the URI address multi-line string pattern.

    The Amazon S3 location of the object.

  • ETagRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Custom string pattern #11.

    The Amazon S3 ETag of the object. Returned by GetTableObjects for validation and used to identify changes to the underlying data.

  • SizeRequired: Number (long).

    The size of the Amazon S3 object in bytes.

  • PartitionValues – An array of UTF-8 strings, not less than 1 or more than 100 strings.

    A list of partition values for the object. A value must be specified for each partition key associated with the table.

    The supported data types are integer, long, date(yyyy-MM-dd), timestamp(yyyy-MM-dd HH:mm:ssXXX or yyyy-MM-dd HH:mm:ss"), string and decimal.

DeleteObjectInput Structure

An object to delete from the governed table.

Fields

  • UriRequired: Uniform resource identifier (uri), not less than 1 or more than 1024 bytes long, matching the URI address multi-line string pattern.

    The Amazon S3 location of the object to delete.

  • ETag – UTF-8 string, not less than 1 or more than 255 bytes long, matching the Custom string pattern #11.

    The Amazon S3 ETag of the object. Returned by GetTableObjects for validation and used to identify changes to the underlying data.

  • PartitionValues – An array of UTF-8 strings, not less than 1 or more than 100 strings.

    A list of partition values for the object. A value must be specified for each partition key associated with the governed table.

WriteOperation Structure

Defines an object to add to or delete from a governed table.

Fields

  • AddObject – An AddObjectInput object.

    A new object to add to the governed table.

  • DeleteObject – A DeleteObjectInput object.

    An object to delete from the governed table.

Operations

GetTableObjects Action (Python: get_table_objects)

Returns the set of Amazon S3 objects that make up the specified governed table. A transaction ID or timestamp can be specified for time-travel queries.

Request

  • CatalogId – Catalog id string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The catalog containing the governed table. Defaults to the caller's account.

  • DatabaseNameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The database containing the governed table.

  • TableNameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The governed table for which to retrieve objects.

  • TransactionId – UTF-8 string, not less than 1 or more than 255 bytes long, matching the Custom string pattern #11.

    The transaction ID at which to read the governed table contents. If this transaction has aborted, an error is returned. If not set, defaults to the most recent committed transaction. Cannot be specified along with QueryAsOfTime.

  • QueryAsOfTime – Timestamp.

    The time as of when to read the governed table contents. If not set, the most recent transaction commit time is used. Cannot be specified along with TransactionId.

  • PartitionPredicate – Predicate string, not more than 2048 bytes long, matching the URI address multi-line string pattern.

    A predicate to filter the objects returned based on the partition keys defined in the governed table.

    • The comparison operators supported are: =, >, <, >=, <=

    • The logical operators supported are: AND

    • The data types supported are integer, long, date(yyyy-MM-dd), timestamp(yyyy-MM-dd HH:mm:ssXXX or yyyy-MM-dd HH:mm:ss"), string and decimal.

  • MaxResults – Number (integer), not less than 1 or more than 1000.

    Specifies how many values to return in a page.

  • NextToken – UTF-8 string, not more than 4096 bytes long.

    A continuation token if this is not the first call to retrieve these objects.

Response

  • Objects – An array of PartitionObjects objects.

    A list of objects organized by partition keys.

  • NextToken – UTF-8 string, not more than 4096 bytes long.

    A continuation token indicating whether additional data is available.

Errors

  • EntityNotFoundException

  • InternalServiceException

  • InvalidInputException

  • OperationTimeoutException

  • TransactionCommittedException

  • TransactionCanceledException

  • ResourceNotReadyException

UpdateTableObjects Action (Python: update_table_objects)

Updates the manifest of Amazon S3 objects that make up the specified governed table.

Request

  • CatalogId – Catalog id string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The catalog containing the governed table to update. Defaults to the caller's account ID.

  • DatabaseNameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The database containing the governed table to update.

  • TableNameRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Single-line string pattern.

    The governed table to update.

  • TransactionIdRequired: UTF-8 string, not less than 1 or more than 255 bytes long, matching the Custom string pattern #11.

    The transaction at which to do the write.

  • WriteOperationsRequired: An array of WriteOperation objects, not less than 1 or more than 100 structures.

    A list of WriteOperation objects that define an object to add to or delete from the manifest for a governed table.

Response

  • No Response parameters.

Errors

  • InternalServiceException

  • InvalidInputException

  • OperationTimeoutException

  • EntityNotFoundException

  • TransactionCommittedException

  • TransactionCanceledException

  • TransactionCommitInProgressException

  • ResourceNotReadyException

  • ConcurrentModificationException