PythonSparkJobExecutableProps

class aws_cdk.aws_glue.PythonSparkJobExecutableProps(*, glue_version, python_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None)

Bases: object

(experimental) Props for creating a Python Spark (ETL or Streaming) job executable.

Parameters:

glue_version (GlueVersion) – (experimental) Glue version.
python_version (PythonVersion) – (experimental) The Python version to use.
script (Code) – (experimental) The script that executes a job.
extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory
extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpath
extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars
extra_python_files (Optional[Sequence[Code]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not set

Stability:

experimental

ExampleMetadata:

infused

Example:

glue.Job(self, "PythonSparkStreamingJob",
    executable=glue.JobExecutable.python_streaming(
        glue_version=glue.GlueVersion.V2_0,
        python_version=glue.PythonVersion.THREE,
        script=glue.Code.from_asset(path.join(__dirname, "job-script/hello_world.py"))
    ),
    description="an example Python Streaming job"
)

Attributes

extra_files

(experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it.

Only individual files are supported, directories are not supported.

Default:: [] - no extra files are copied to the working directory
See:: --extra-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
Stability:: experimental

extra_jars

(experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported.

Default:: [] - no extra jars are added to the classpath
See:: --extra-jars in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
Stability:: experimental

extra_jars_first

(experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath.

Default:: false - priority is not given to user-provided jars
See:: --user-jars-first in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
Stability:: experimental

extra_python_files

(experimental) Additional Python files that AWS Glue adds to the Python path before executing your script.

Only individual files are supported, directories are not supported.

Default:

no extra python files and argument is not set

See:

--extra-py-files in https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability:

experimental

glue_version

(experimental) Glue version.

See:: https://docs.aws.amazon.com/glue/latest/dg/release-notes.html
Stability:: experimental

python_version

(experimental) The Python version to use.

Stability:: experimental

script

(experimental) The script that executes a job.

Stability:: experimental