Join Class - AWS Glue

Join Class

Performs an equality join on two DynamicFrames.

Methods

__call__(frame1, frame2, keys1, keys2, transformation_ctx = "")

Performs an equality join on two DynamicFrames.

  • frame1 – The first DynamicFrame to join (required).

  • frame2 – The second DynamicFrame to join (required).

  • keys1 – The keys to join on for the first frame (required).

  • keys2 – The keys to join on for the second frame (required).

  • transformation_ctx – A unique string that is used to identify state information (optional).

Returns a new DynamicFrame obtained by joining the two DynamicFrames.

apply(cls, *args, **kwargs)

Inherited from GlueTransform apply

name(cls)

Inherited from GlueTransform name

describeArgs(cls)

Inherited from GlueTransform describeArgs

describeReturn(cls)

Inherited from GlueTransform describeReturn

describeTransform(cls)

Inherited from GlueTransform describeTransform

describeErrors(cls)

Inherited from GlueTransform describeErrors

describe(cls)

Inherited from GlueTransform describe

Example for Join

The Join function manages duplicate columns. Each dataset has a column named zip. AWS Glue adds a period (.) to one of the duplicate column names to avoid errors.

dyf_join = Join.apply(dyf_json, dyf_selectFields, 'zip', 'zip') dyf_join.toDF().show() +--------------------+-----+-----+ | customers| .zip| zip| +--------------------+-----+-----+ |[[108 Park Street...|75091|75091| |[[66 P Street, NY...|75023|75023| |[[708 Fed Ln, CA,...|90093|90093| +--------------------+-----+-----+