JobProps

class aws_cdk.aws_glue.JobProps(*, executable, connections=None, continuous_logging=None, default_arguments=None, description=None, enable_profiling_metrics=None, job_name=None, max_capacity=None, max_concurrent_runs=None, max_retries=None, notify_delay_after=None, role=None, security_configuration=None, spark_ui=None, tags=None, timeout=None, worker_count=None, worker_type=None)

Bases: object

(experimental) Construction properties for {@link Job}.

Parameters
  • executable (JobExecutable) – (experimental) The job’s executable properties.

  • connections (Optional[Sequence[IConnection]]) – (experimental) The {@link Connection}s used for this job. Connections are used to connect to other AWS Service or resources within a VPC. Default: [] - no connections are added to the job

  • continuous_logging (Optional[ContinuousLoggingProps]) – (experimental) Enables continuous logging with the specified props. Default: - continuous logging is disabled.

  • default_arguments (Optional[Mapping[str, str]]) – (experimental) The default arguments for this job, specified as name-value pairs. Default: - no arguments

  • description (Optional[str]) – (experimental) The description of the job. Default: - no value

  • enable_profiling_metrics (Optional[bool]) – (experimental) Enables the collection of metrics for job profiling. Default: - no profiling metrics emitted.

  • job_name (Optional[str]) – (experimental) The name of the job. Default: - a name is automatically generated

  • max_capacity (Union[int, float, None]) – (experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead. Default: - 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell

  • max_concurrent_runs (Union[int, float, None]) – (experimental) The maximum number of concurrent runs allowed for the job. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit. Default: 1

  • max_retries (Union[int, float, None]) – (experimental) The maximum number of times to retry this job after a job run fails. Default: 0

  • notify_delay_after (Optional[Duration]) – (experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification. Default: - no delay notifications

  • role (Optional[IRole]) – (experimental) The IAM role assumed by Glue to run this job. If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions. Default: - a role is automatically generated

  • security_configuration (Optional[ISecurityConfiguration]) – (experimental) The {@link SecurityConfiguration} to use for this job. Default: - no security configuration.

  • spark_ui (Optional[SparkUIProps]) – (experimental) Enables the Spark UI debugging and monitoring with the specified props. Default: - Spark UI debugging and monitoring is disabled.

  • tags (Optional[Mapping[str, str]]) – (experimental) The tags to add to the resources on which the job runs. Default: {} - no tags

  • timeout (Optional[Duration]) – (experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. Default: cdk.Duration.hours(48)

  • worker_count (Union[int, float, None]) – (experimental) The number of workers of a defined {@link WorkerType} that are allocated when a job runs. Default: - differs based on specific Glue version/worker type

  • worker_type (Optional[WorkerType]) – (experimental) The type of predefined worker that is allocated when a job runs. Default: - differs based on specific Glue version

Stability

experimental

Attributes

connections

(experimental) The {@link Connection}s used for this job.

Connections are used to connect to other AWS Service or resources within a VPC.

Default

[] - no connections are added to the job

Stability

experimental

Return type

Optional[List[IConnection]]

continuous_logging

(experimental) Enables continuous logging with the specified props.

Default
  • continuous logging is disabled.

See

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[ContinuousLoggingProps]

default_arguments

(experimental) The default arguments for this job, specified as name-value pairs.

Default
  • no arguments

See

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters

Stability

experimental

Return type

Optional[Mapping[str, str]]

description

(experimental) The description of the job.

Default
  • no value

Stability

experimental

Return type

Optional[str]

enable_profiling_metrics

(experimental) Enables the collection of metrics for job profiling.

Default
  • no profiling metrics emitted.

See

--enable-metrics at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[bool]

executable

(experimental) The job’s executable properties.

Stability

experimental

Return type

JobExecutable

job_name

(experimental) The name of the job.

Default
  • a name is automatically generated

Stability

experimental

Return type

Optional[str]

max_capacity

(experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.

Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.

Default
  • 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell

Stability

experimental

Return type

Union[int, float, None]

max_concurrent_runs

(experimental) The maximum number of concurrent runs allowed for the job.

An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.

Default

1

Stability

experimental

Return type

Union[int, float, None]

max_retries

(experimental) The maximum number of times to retry this job after a job run fails.

Default

0

Stability

experimental

Return type

Union[int, float, None]

notify_delay_after

(experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification.

Default
  • no delay notifications

Stability

experimental

Return type

Optional[Duration]

role

(experimental) The IAM role assumed by Glue to run this job.

If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.

Default
  • a role is automatically generated

See

https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html

Stability

experimental

Return type

Optional[IRole]

security_configuration

(experimental) The {@link SecurityConfiguration} to use for this job.

Default
  • no security configuration.

Stability

experimental

Return type

Optional[ISecurityConfiguration]

spark_ui

(experimental) Enables the Spark UI debugging and monitoring with the specified props.

Default
  • Spark UI debugging and monitoring is disabled.

See

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[SparkUIProps]

tags

(experimental) The tags to add to the resources on which the job runs.

Default

{} - no tags

Stability

experimental

Return type

Optional[Mapping[str, str]]

timeout

(experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.

Default

cdk.Duration.hours(48)

Stability

experimental

Return type

Optional[Duration]

worker_count

(experimental) The number of workers of a defined {@link WorkerType} that are allocated when a job runs.

Default
  • differs based on specific Glue version/worker type

Stability

experimental

Return type

Union[int, float, None]

worker_type

(experimental) The type of predefined worker that is allocated when a job runs.

Default
  • differs based on specific Glue version

Stability

experimental

Return type

Optional[WorkerType]