SparkExtraCodeProps

class aws_cdk.aws_glue_alpha.SparkExtraCodeProps(*, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None)

Bases: object

(experimental) Code props for different {@link Code} assets used by different types of Spark jobs.

Parameters:

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Default: - no extra files specified.

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located. Default: - no extra jar files

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars

  • extra_python_files (Optional[Sequence[Code]]) – (experimental) Extra Python Files S3 URL (optional) S3 URL where additional python dependencies are located. Default: - no extra files

Stability:

experimental

ExampleMetadata:

fixture=_generated

Example:

# The code below shows an example of how to instantiate this type.
# The values are placeholders you should change.
import aws_cdk.aws_glue_alpha as glue_alpha

# code: glue_alpha.Code

spark_extra_code_props = glue_alpha.SparkExtraCodeProps(
    extra_files=[code],
    extra_jars=[code],
    extra_jars_first=False,
    extra_python_files=[code]
)

Attributes

extra_files

(experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.

Default:

  • no extra files specified.

See:

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability:

experimental

extra_jars

(experimental) Extra Jars S3 URL (optional) S3 URL where additional jar dependencies are located.

Default:

  • no extra jar files

Stability:

experimental

extra_jars_first

(experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath.

Default:

false - priority is not given to user-provided jars

See:

--user-jars-first in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability:

experimental

extra_python_files

(experimental) Extra Python Files S3 URL (optional) S3 URL where additional python dependencies are located.

Default:

  • no extra files

Stability:

experimental