Relationalize Class - AWS Glue
 — methods —__call__applynamedescribeArgsdescribeReturndescribeTransformdescribeErrorsdescribeExample

Relationalize Class

Flattens nested schema in a DynamicFrame and pivots out array columns from the flattened frame.

Methods

__call__(frame, staging_path=None, name='roottable', options=None, transformation_ctx = "", info = "", stageThreshold = 0, totalThreshold = 0)

Relationalizes a DynamicFrame and produces a list of frames that are generated by unnesting nested columns and pivoting array columns. The pivoted array column can be joined to the root table using the joinkey generated in the unnest phase.

  • frame – The DynamicFrame to relationalize (required).

  • staging_path – The path at which to store partitions of pivoted tables in CSV format (optional). Pivoted tables are read back from this path.

  • name – The name of the root table (optional).

  • options – A dictionary of optional parameters. Currently unused.

  • transformation_ctx – A unique string that is used to identify state information (optional).

  • info – A string associated with errors in the transformation (optional).

  • stageThreshold – The maximum number of errors that can occur in the transformation before it errors out (optional; the default is zero).

  • totalThreshold – The maximum number of errors that can occur overall before processing errors out (optional; the default is zero).

Return a DynamicFrameCollection containing the DynamicFrames produced by from the relationalize operation.

apply(cls, *args, **kwargs)

Inherited from GlueTransform apply.

name(cls)

Inherited from GlueTransform name.

describeArgs(cls)

Inherited from GlueTransform describeArgs.

describeReturn(cls)

Inherited from GlueTransform describeReturn.

describeTransform(cls)

Inherited from GlueTransform describeTransform.

describeErrors(cls)

Inherited from GlueTransform describeErrors.

describe(cls)

Inherited from GlueTransform describe.

Example for Relationalize

The Relationalize function can flatten nested structures and create multiple dynamic frames. In this example, the customer column is a nested structure, and Relationalize converts it into multiple flattened DynamicFrames.

dyf_relationize = dyf_orders.relationalize("root", "/home/glue/GlueLocalOutput")

The result is a collection of DynamicFrames. Use the SelectFromCollection function to retrieve a specific DynamicFrame from the result.