JobExecutable
- class aws_cdk.aws_glue_alpha.JobExecutable(*args: Any, **kwargs)
Bases:
object
(experimental) The executable properties related to the Glue job’s GlueVersion, JobType and code.
- Stability:
experimental
- ExampleMetadata:
infused
Example:
glue.Job(self, "EnableSparkUI", job_name="EtlJobWithSparkUIPrefix", spark_uI=glue.SparkUIProps( enabled=True ), executable=glue.JobExecutable.python_etl( glue_version=glue.GlueVersion.V3_0, python_version=glue.PythonVersion.THREE, script=glue.Code.from_asset(path.join(__dirname, "job-script/hello_world.py")) ) )
Methods
- bind()
(experimental) Called during Job initialization to get JobExecutableConfig.
- Stability:
experimental
- Return type:
Static Methods
- classmethod of(*, glue_version, language, script, type, class_name=None, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None, python_version=None, runtime=None)
(experimental) Create a custom JobExecutable.
- Parameters:
glue_version (
GlueVersion
) – (experimental) Glue version.language (
JobLanguage
) – (experimental) The language of the job (Scala or Python).script (
Code
) – (experimental) The script that is executed by a job.type (
JobType
) – (experimental) Specify the type of the job whether it’s an Apache Spark ETL or streaming one or if it’s a Python shell job.class_name (
Optional
[str
]) – (experimental) The Scala class that serves as the entry point for the job. This applies only if your the job langauage is Scala. Default: - no scala className specifiedextra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Default: - no extra files specified.extra_jars (
Optional
[Sequence
[Code
]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Default: - no extra jars specified.extra_jars_first (
Optional
[bool
]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: - extra jars are not prioritized.extra_python_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Default: - no extra python files specified.python_version (
Optional
[PythonVersion
]) – (experimental) The Python version to use. Default: - no python version specifiedruntime (
Optional
[Runtime
]) – (experimental) The Runtime to use. Default: - no runtime specified
- Stability:
experimental
- Return type:
- classmethod python_etl(*, glue_version, python_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None, runtime=None)
(experimental) Create Python executable props for Apache Spark ETL job.
- Parameters:
glue_version (
GlueVersion
) – (experimental) Glue version.python_version (
PythonVersion
) – (experimental) The Python version to use.script (
Code
) – (experimental) The script that executes a job.extra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directoryextra_jars (
Optional
[Sequence
[Code
]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpathextra_jars_first (
Optional
[bool
]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jarsextra_python_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not setruntime (
Optional
[Runtime
]) – (experimental) Runtime. It is required for Ray jobs.
- Stability:
experimental
- Return type:
- classmethod python_ray(*, glue_version, python_version, script, extra_files=None, extra_python_files=None, runtime=None)
(experimental) Create Python executable props for Ray jobs.
- Parameters:
glue_version (
GlueVersion
) – (experimental) Glue version.python_version (
PythonVersion
) – (experimental) The Python version to use.script (
Code
) – (experimental) The script that executes a job.extra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directoryextra_python_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not setruntime (
Optional
[Runtime
]) – (experimental) Runtime. It is required for Ray jobs.
- Stability:
experimental
- Return type:
- classmethod python_shell(*, glue_version, python_version, script, extra_files=None, extra_python_files=None, runtime=None)
(experimental) Create Python executable props for python shell jobs.
- Parameters:
glue_version (
GlueVersion
) – (experimental) Glue version.python_version (
PythonVersion
) – (experimental) The Python version to use.script (
Code
) – (experimental) The script that executes a job.extra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directoryextra_python_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not setruntime (
Optional
[Runtime
]) – (experimental) Runtime. It is required for Ray jobs.
- Stability:
experimental
- Return type:
- classmethod python_streaming(*, glue_version, python_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None, runtime=None)
(experimental) Create Python executable props for Apache Spark Streaming job.
- Parameters:
glue_version (
GlueVersion
) – (experimental) Glue version.python_version (
PythonVersion
) – (experimental) The Python version to use.script (
Code
) – (experimental) The script that executes a job.extra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directoryextra_jars (
Optional
[Sequence
[Code
]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpathextra_jars_first (
Optional
[bool
]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jarsextra_python_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not setruntime (
Optional
[Runtime
]) – (experimental) Runtime. It is required for Ray jobs.
- Stability:
experimental
- Return type:
- classmethod scala_etl(*, class_name, glue_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, runtime=None)
(experimental) Create Scala executable props for Apache Spark ETL job.
- Parameters:
class_name (
str
) – (experimental) The fully qualified Scala class name that serves as the entry point for the job.glue_version (
GlueVersion
) – (experimental) Glue version.script (
Code
) – (experimental) The script that executes a job.extra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directoryextra_jars (
Optional
[Sequence
[Code
]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpathextra_jars_first (
Optional
[bool
]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jarsruntime (
Optional
[Runtime
]) – (experimental) Runtime. It is required for Ray jobs.
- Stability:
experimental
- Return type:
- classmethod scala_streaming(*, class_name, glue_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, runtime=None)
(experimental) Create Scala executable props for Apache Spark Streaming job.
- Parameters:
class_name (
str
) – (experimental) The fully qualified Scala class name that serves as the entry point for the job.glue_version (
GlueVersion
) – (experimental) Glue version.script (
Code
) – (experimental) The script that executes a job.extra_files (
Optional
[Sequence
[Code
]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directoryextra_jars (
Optional
[Sequence
[Code
]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpathextra_jars_first (
Optional
[bool
]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jarsruntime (
Optional
[Runtime
]) – (experimental) Runtime. It is required for Ray jobs.
- Stability:
experimental
- Return type: