PythonSparkJobExecutableProps

class aws_cdk.aws_glue.PythonSparkJobExecutableProps(*, glue_version, python_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None)

Bases: object

(experimental) Props for creating a Python Spark (ETL or Streaming) job executable.

Parameters
  • glue_version (GlueVersion) – (experimental) Glue version.

  • python_version (PythonVersion) – (experimental) The Python version to use.

  • script (Code) – (experimental) The script that executes a job.

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpath

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars

  • extra_python_files (Optional[Sequence[Code]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not set

Stability

experimental

Attributes

extra_files

(experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.

Only individual files are supported, directories are not supported.

Default

[] - no extra files are copied to the working directory

See

--extra-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[List[Code]]

extra_jars

(experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported.

Default

[] - no extra jars are added to the classpath

See

--extra-jars in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[List[Code]]

extra_jars_first

(experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath.

Default

false - priority is not given to user-provided jars

See

--user-jars-first in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[bool]

extra_python_files

(experimental) Additional Python files that AWS Glue adds to the Python path before executing your script.

Only individual files are supported, directories are not supported.

Default
  • no extra python files and argument is not set

See

--extra-py-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[List[Code]]

glue_version

(experimental) Glue version.

See

https://docs.aws.amazon.com/glue/latest/dg/release-notes.html

Stability

experimental

Return type

GlueVersion

python_version

(experimental) The Python version to use.

Stability

experimental

Return type

PythonVersion

script

(experimental) The script that executes a job.

Stability

experimental

Return type

Code