JobExecutable

class aws_cdk.aws_glue.JobExecutable(*args: Any, **kwargs)

Bases: object

(experimental) The executable properties related to the Glue job’s GlueVersion, JobType and code.

Stability

experimental

Example:

# bucket is of type Bucket

glue.Job(self, "PythonShellJob",
    executable=glue.JobExecutable.python_shell(
        glue_version=glue.GlueVersion.V1_0,
        python_version=glue.PythonVersion.THREE,
        script=glue.Code.from_bucket(bucket, "script.py")
    ),
    description="an example Python Shell job"
)

Methods

bind()

(experimental) Called during Job initialization to get JobExecutableConfig.

Stability

experimental

Return type

JobExecutableConfig

Static Methods

classmethod of(*, class_name=None, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None, glue_version, language, python_version=None, script, type)

(experimental) Create a custom JobExecutable.

Parameters
  • class_name (Optional[str]) – (experimental) The Scala class that serves as the entry point for the job. This applies only if your the job langauage is Scala. Default: - no scala className specified

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Default: - no extra files specified.

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Default: - no extra jars specified.

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: - extra jars are not prioritized.

  • extra_python_files (Optional[Sequence[Code]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Default: - no extra python files specified.

  • glue_version (GlueVersion) – (experimental) Glue version.

  • language (JobLanguage) – (experimental) The language of the job (Scala or Python).

  • python_version (Optional[PythonVersion]) – (experimental) The Python version to use. Default: - no python version specified

  • script (Code) – (experimental) The script that is executed by a job.

  • type (JobType) – (experimental) Specify the type of the job whether it’s an Apache Spark ETL or streaming one or if it’s a Python shell job.

Stability

experimental

Return type

JobExecutable

classmethod python_etl(*, glue_version, python_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None)

(experimental) Create Python executable props for Apache Spark ETL job.

Parameters
  • glue_version (GlueVersion) – (experimental) Glue version.

  • python_version (PythonVersion) – (experimental) The Python version to use.

  • script (Code) – (experimental) The script that executes a job.

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpath

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars

  • extra_python_files (Optional[Sequence[Code]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not set

Stability

experimental

Return type

JobExecutable

classmethod python_shell(*, glue_version, python_version, script, extra_files=None, extra_python_files=None)

(experimental) Create Python executable props for python shell jobs.

Parameters
  • glue_version (GlueVersion) – (experimental) Glue version.

  • python_version (PythonVersion) – (experimental) The Python version to use.

  • script (Code) – (experimental) The script that executes a job.

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory

  • extra_python_files (Optional[Sequence[Code]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not set

Stability

experimental

Return type

JobExecutable

classmethod python_streaming(*, glue_version, python_version, script, extra_files=None, extra_jars=None, extra_jars_first=None, extra_python_files=None)

(experimental) Create Python executable props for Apache Spark Streaming job.

Parameters
  • glue_version (GlueVersion) – (experimental) Glue version.

  • python_version (PythonVersion) – (experimental) The Python version to use.

  • script (Code) – (experimental) The script that executes a job.

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpath

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars

  • extra_python_files (Optional[Sequence[Code]]) – (experimental) Additional Python files that AWS Glue adds to the Python path before executing your script. Only individual files are supported, directories are not supported. Default: - no extra python files and argument is not set

Stability

experimental

Return type

JobExecutable

classmethod scala_etl(*, class_name, glue_version, script, extra_files=None, extra_jars=None, extra_jars_first=None)

(experimental) Create Scala executable props for Apache Spark ETL job.

Parameters
  • class_name (str) – (experimental) The fully qualified Scala class name that serves as the entry point for the job.

  • glue_version (GlueVersion) – (experimental) Glue version.

  • script (Code) – (experimental) The script that executes a job.

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpath

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars

Stability

experimental

Return type

JobExecutable

classmethod scala_streaming(*, class_name, glue_version, script, extra_files=None, extra_jars=None, extra_jars_first=None)

(experimental) Create Scala executable props for Apache Spark Streaming job.

Parameters
  • class_name (str) – (experimental) The fully qualified Scala class name that serves as the entry point for the job.

  • glue_version (GlueVersion) – (experimental) Glue version.

  • script (Code) – (experimental) The script that executes a job.

  • extra_files (Optional[Sequence[Code]]) – (experimental) Additional files, such as configuration files that AWS Glue copies to the working directory of your script before executing it. Only individual files are supported, directories are not supported. Default: [] - no extra files are copied to the working directory

  • extra_jars (Optional[Sequence[Code]]) – (experimental) Additional Java .jar files that AWS Glue adds to the Java classpath before executing your script. Only individual files are supported, directories are not supported. Default: [] - no extra jars are added to the classpath

  • extra_jars_first (Optional[bool]) – (experimental) Setting this value to true prioritizes the customer’s extra JAR files in the classpath. Default: false - priority is not given to user-provided jars

Stability

experimental

Return type

JobExecutable