Job

class aws_cdk.aws_glue.Job(scope, id, *, executable, connections=None, continuous_logging=None, default_arguments=None, description=None, enable_profiling_metrics=None, job_name=None, max_capacity=None, max_concurrent_runs=None, max_retries=None, notify_delay_after=None, role=None, security_configuration=None, spark_ui=None, tags=None, timeout=None, worker_count=None, worker_type=None)

Bases: aws_cdk.core.Resource

(experimental) A Glue Job.

Stability

experimental

Parameters
  • scope (Construct) –

  • id (str) –

  • executable (JobExecutable) – (experimental) The job’s executable properties.

  • connections (Optional[Sequence[IConnection]]) – (experimental) The {@link Connection}s used for this job. Connections are used to connect to other AWS Service or resources within a VPC. Default: [] - no connections are added to the job

  • continuous_logging (Optional[ContinuousLoggingProps]) – (experimental) Enables continuous logging with the specified props. Default: - continuous logging is disabled.

  • default_arguments (Optional[Mapping[str, str]]) – (experimental) The default arguments for this job, specified as name-value pairs. Default: - no arguments

  • description (Optional[str]) – (experimental) The description of the job. Default: - no value

  • enable_profiling_metrics (Optional[bool]) – (experimental) Enables the collection of metrics for job profiling. Default: - no profiling metrics emitted.

  • job_name (Optional[str]) – (experimental) The name of the job. Default: - a name is automatically generated

  • max_capacity (Union[int, float, None]) – (experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead. Default: - 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell

  • max_concurrent_runs (Union[int, float, None]) – (experimental) The maximum number of concurrent runs allowed for the job. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit. Default: 1

  • max_retries (Union[int, float, None]) – (experimental) The maximum number of times to retry this job after a job run fails. Default: 0

  • notify_delay_after (Optional[Duration]) – (experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification. Default: - no delay notifications

  • role (Optional[IRole]) – (experimental) The IAM role assumed by Glue to run this job. If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions. Default: - a role is automatically generated

  • security_configuration (Optional[ISecurityConfiguration]) – (experimental) The {@link SecurityConfiguration} to use for this job. Default: - no security configuration.

  • spark_ui (Optional[SparkUIProps]) – (experimental) Enables the Spark UI debugging and monitoring with the specified props. Default: - Spark UI debugging and monitoring is disabled.

  • tags (Optional[Mapping[str, str]]) – (experimental) The tags to add to the resources on which the job runs. Default: {} - no tags

  • timeout (Optional[Duration]) – (experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. Default: cdk.Duration.hours(48)

  • worker_count (Union[int, float, None]) – (experimental) The number of workers of a defined {@link WorkerType} that are allocated when a job runs. Default: - differs based on specific Glue version/worker type

  • worker_type (Optional[WorkerType]) – (experimental) The type of predefined worker that is allocated when a job runs. Default: - differs based on specific Glue version

Stability

experimental

Methods

apply_removal_policy(policy)

Apply the given removal policy to this resource.

The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you’ve removed it from the CDK application or because you’ve made a change that requires the resource to be replaced.

The resource can be deleted (RemovalPolicy.DESTROY), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN).

Parameters

policy (RemovalPolicy) –

Return type

None

metric(metric_name, type, *, account=None, color=None, dimensions=None, dimensions_map=None, label=None, period=None, region=None, statistic=None, unit=None)

(experimental) Create a CloudWatch metric.

Parameters
  • metric_name (str) – name of the metric typically prefixed with glue.driver., glue.<executorId>. or glue.ALL..

  • type (MetricType) – the metric type.

  • account (Optional[str]) – Account which this metric comes from. Default: - Deployment account.

  • color (Optional[str]) – The hex color code, prefixed with ‘#’ (e.g. ‘#00ff00’), to use when this metric is rendered on a graph. The Color class has a set of standard colors that can be used here. Default: - Automatic color

  • dimensions (Optional[Mapping[str, Any]]) – (deprecated) Dimensions of the metric. Default: - No dimensions.

  • dimensions_map (Optional[Mapping[str, str]]) – Dimensions of the metric. Default: - No dimensions.

  • label (Optional[str]) – Label for this metric when added to a Graph in a Dashboard. Default: - No label

  • period (Optional[Duration]) – The period over which the specified statistic is applied. Default: Duration.minutes(5)

  • region (Optional[str]) – Region which this metric comes from. Default: - Deployment region.

  • statistic (Optional[str]) – What function to use for aggregating. Can be one of the following: - “Minimum” | “min” - “Maximum” | “max” - “Average” | “avg” - “Sum” | “sum” - “SampleCount | “n” - “pNN.NN” Default: Average

  • unit (Optional[Unit]) – Unit used to filter the metric stream. Only refer to datums emitted to the metric stream with the given unit and ignore all others. Only useful when datums are being emitted to the same metric stream under different units. The default is to use all matric datums in the stream, regardless of unit, which is recommended in nearly all cases. CloudWatch does not honor this property for graphs. Default: - All metric datums in the given metric stream

See

https://docs.aws.amazon.com/glue/latest/dg/monitoring-awsglue-with-cloudwatch-metrics.html

Stability

experimental

Return type

Metric

metric_failure(*, account=None, color=None, dimensions=None, dimensions_map=None, label=None, period=None, region=None, statistic=None, unit=None)

(experimental) Return a CloudWatch Metric indicating job failure.

This metric is based on the Rule returned by no-args onFailure() call.

Parameters
  • account (Optional[str]) – Account which this metric comes from. Default: - Deployment account.

  • color (Optional[str]) – The hex color code, prefixed with ‘#’ (e.g. ‘#00ff00’), to use when this metric is rendered on a graph. The Color class has a set of standard colors that can be used here. Default: - Automatic color

  • dimensions (Optional[Mapping[str, Any]]) – (deprecated) Dimensions of the metric. Default: - No dimensions.

  • dimensions_map (Optional[Mapping[str, str]]) – Dimensions of the metric. Default: - No dimensions.

  • label (Optional[str]) – Label for this metric when added to a Graph in a Dashboard. Default: - No label

  • period (Optional[Duration]) – The period over which the specified statistic is applied. Default: Duration.minutes(5)

  • region (Optional[str]) – Region which this metric comes from. Default: - Deployment region.

  • statistic (Optional[str]) – What function to use for aggregating. Can be one of the following: - “Minimum” | “min” - “Maximum” | “max” - “Average” | “avg” - “Sum” | “sum” - “SampleCount | “n” - “pNN.NN” Default: Average

  • unit (Optional[Unit]) – Unit used to filter the metric stream. Only refer to datums emitted to the metric stream with the given unit and ignore all others. Only useful when datums are being emitted to the same metric stream under different units. The default is to use all matric datums in the stream, regardless of unit, which is recommended in nearly all cases. CloudWatch does not honor this property for graphs. Default: - All metric datums in the given metric stream

Stability

experimental

Return type

Metric

metric_success(*, account=None, color=None, dimensions=None, dimensions_map=None, label=None, period=None, region=None, statistic=None, unit=None)

(experimental) Return a CloudWatch Metric indicating job success.

This metric is based on the Rule returned by no-args onSuccess() call.

Parameters
  • account (Optional[str]) – Account which this metric comes from. Default: - Deployment account.

  • color (Optional[str]) – The hex color code, prefixed with ‘#’ (e.g. ‘#00ff00’), to use when this metric is rendered on a graph. The Color class has a set of standard colors that can be used here. Default: - Automatic color

  • dimensions (Optional[Mapping[str, Any]]) – (deprecated) Dimensions of the metric. Default: - No dimensions.

  • dimensions_map (Optional[Mapping[str, str]]) – Dimensions of the metric. Default: - No dimensions.

  • label (Optional[str]) – Label for this metric when added to a Graph in a Dashboard. Default: - No label

  • period (Optional[Duration]) – The period over which the specified statistic is applied. Default: Duration.minutes(5)

  • region (Optional[str]) – Region which this metric comes from. Default: - Deployment region.

  • statistic (Optional[str]) – What function to use for aggregating. Can be one of the following: - “Minimum” | “min” - “Maximum” | “max” - “Average” | “avg” - “Sum” | “sum” - “SampleCount | “n” - “pNN.NN” Default: Average

  • unit (Optional[Unit]) – Unit used to filter the metric stream. Only refer to datums emitted to the metric stream with the given unit and ignore all others. Only useful when datums are being emitted to the same metric stream under different units. The default is to use all matric datums in the stream, regardless of unit, which is recommended in nearly all cases. CloudWatch does not honor this property for graphs. Default: - All metric datums in the given metric stream

Stability

experimental

Return type

Metric

metric_timeout(*, account=None, color=None, dimensions=None, dimensions_map=None, label=None, period=None, region=None, statistic=None, unit=None)

(experimental) Return a CloudWatch Metric indicating job timeout.

This metric is based on the Rule returned by no-args onTimeout() call.

Parameters
  • account (Optional[str]) – Account which this metric comes from. Default: - Deployment account.

  • color (Optional[str]) – The hex color code, prefixed with ‘#’ (e.g. ‘#00ff00’), to use when this metric is rendered on a graph. The Color class has a set of standard colors that can be used here. Default: - Automatic color

  • dimensions (Optional[Mapping[str, Any]]) – (deprecated) Dimensions of the metric. Default: - No dimensions.

  • dimensions_map (Optional[Mapping[str, str]]) – Dimensions of the metric. Default: - No dimensions.

  • label (Optional[str]) – Label for this metric when added to a Graph in a Dashboard. Default: - No label

  • period (Optional[Duration]) – The period over which the specified statistic is applied. Default: Duration.minutes(5)

  • region (Optional[str]) – Region which this metric comes from. Default: - Deployment region.

  • statistic (Optional[str]) – What function to use for aggregating. Can be one of the following: - “Minimum” | “min” - “Maximum” | “max” - “Average” | “avg” - “Sum” | “sum” - “SampleCount | “n” - “pNN.NN” Default: Average

  • unit (Optional[Unit]) – Unit used to filter the metric stream. Only refer to datums emitted to the metric stream with the given unit and ignore all others. Only useful when datums are being emitted to the same metric stream under different units. The default is to use all matric datums in the stream, regardless of unit, which is recommended in nearly all cases. CloudWatch does not honor this property for graphs. Default: - All metric datums in the given metric stream

Stability

experimental

Return type

Metric

on_event(id, *, description=None, event_pattern=None, rule_name=None, target=None)

(experimental) Create a CloudWatch Event Rule for this Glue Job when it’s in a given state.

Parameters
  • id (str) – construct id.

  • description (Optional[str]) – A description of the rule’s purpose. Default: - No description

  • event_pattern (Optional[EventPattern]) – Additional restrictions for the event to route to the specified target. The method that generates the rule probably imposes some type of event filtering. The filtering implied by what you pass here is added on top of that filtering. Default: - No additional filtering based on an event pattern.

  • rule_name (Optional[str]) – A name for the rule. Default: AWS CloudFormation generates a unique physical ID.

  • target (Optional[IRuleTarget]) – The target to register for the event. Default: - No target is added to the rule. Use addTarget() to add a target.

See

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/EventTypes.html#glue-event-types

Stability

experimental

Return type

Rule

on_failure(id, *, description=None, event_pattern=None, rule_name=None, target=None)

(experimental) Return a CloudWatch Event Rule matching FAILED state.

Parameters
  • id (str) – construct id.

  • description (Optional[str]) – A description of the rule’s purpose. Default: - No description

  • event_pattern (Optional[EventPattern]) – Additional restrictions for the event to route to the specified target. The method that generates the rule probably imposes some type of event filtering. The filtering implied by what you pass here is added on top of that filtering. Default: - No additional filtering based on an event pattern.

  • rule_name (Optional[str]) – A name for the rule. Default: AWS CloudFormation generates a unique physical ID.

  • target (Optional[IRuleTarget]) – The target to register for the event. Default: - No target is added to the rule. Use addTarget() to add a target.

Stability

experimental

Return type

Rule

on_state_change(id, job_state, *, description=None, event_pattern=None, rule_name=None, target=None)

(experimental) Create a CloudWatch Event Rule for the transition into the input jobState.

Parameters
  • id (str) – construct id.

  • job_state (JobState) – the job state.

  • description (Optional[str]) – A description of the rule’s purpose. Default: - No description

  • event_pattern (Optional[EventPattern]) – Additional restrictions for the event to route to the specified target. The method that generates the rule probably imposes some type of event filtering. The filtering implied by what you pass here is added on top of that filtering. Default: - No additional filtering based on an event pattern.

  • rule_name (Optional[str]) – A name for the rule. Default: AWS CloudFormation generates a unique physical ID.

  • target (Optional[IRuleTarget]) – The target to register for the event. Default: - No target is added to the rule. Use addTarget() to add a target.

Stability

experimental

Return type

Rule

on_success(id, *, description=None, event_pattern=None, rule_name=None, target=None)

(experimental) Create a CloudWatch Event Rule matching JobState.SUCCEEDED.

Parameters
  • id (str) – construct id.

  • description (Optional[str]) – A description of the rule’s purpose. Default: - No description

  • event_pattern (Optional[EventPattern]) – Additional restrictions for the event to route to the specified target. The method that generates the rule probably imposes some type of event filtering. The filtering implied by what you pass here is added on top of that filtering. Default: - No additional filtering based on an event pattern.

  • rule_name (Optional[str]) – A name for the rule. Default: AWS CloudFormation generates a unique physical ID.

  • target (Optional[IRuleTarget]) – The target to register for the event. Default: - No target is added to the rule. Use addTarget() to add a target.

Stability

experimental

Return type

Rule

on_timeout(id, *, description=None, event_pattern=None, rule_name=None, target=None)

(experimental) Return a CloudWatch Event Rule matching TIMEOUT state.

Parameters
  • id (str) – construct id.

  • description (Optional[str]) – A description of the rule’s purpose. Default: - No description

  • event_pattern (Optional[EventPattern]) – Additional restrictions for the event to route to the specified target. The method that generates the rule probably imposes some type of event filtering. The filtering implied by what you pass here is added on top of that filtering. Default: - No additional filtering based on an event pattern.

  • rule_name (Optional[str]) – A name for the rule. Default: AWS CloudFormation generates a unique physical ID.

  • target (Optional[IRuleTarget]) – The target to register for the event. Default: - No target is added to the rule. Use addTarget() to add a target.

Stability

experimental

Return type

Rule

to_string()

Returns a string representation of this construct.

Return type

str

Attributes

env

The environment this resource belongs to.

For resources that are created and managed by the CDK (generally, those created by creating new class instances like Role, Bucket, etc.), this is always the same as the environment of the stack they belong to; however, for imported resources (those obtained from static methods like fromRoleArn, fromBucketName, etc.), that might be different than the stack they were imported into.

Return type

ResourceEnvironment

grant_principal

(experimental) The principal this Glue Job is running as.

Stability

experimental

Return type

IPrincipal

job_arn

(experimental) The ARN of the job.

Stability

experimental

Return type

str

job_name

(experimental) The name of the job.

Stability

experimental

Return type

str

node

The construct tree node associated with this construct.

Return type

ConstructNode

role

(experimental) The IAM role Glue assumes to run this job.

Stability

experimental

Return type

IRole

spark_ui_logging_location

(experimental) The Spark UI logs location if Spark UI monitoring and debugging is enabled.

See

https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Stability

experimental

Return type

Optional[SparkUILoggingLocation]

stack

The stack in which this resource is defined.

Return type

Stack

Static Methods

classmethod from_job_attributes(scope, id, *, job_name, role=None)

(experimental) Creates a Glue Job.

Parameters
  • scope (Construct) – The scope creating construct (usually this).

  • id (str) – The construct’s id.

  • job_name (str) – (experimental) The name of the job.

  • role (Optional[IRole]) – (experimental) The IAM role assumed by Glue to run this job. Default: - undefined

Stability

experimental

Return type

IJob

classmethod is_construct(x)

Return whether the given object is a Construct.

Parameters

x (Any) –

Return type

bool

classmethod is_resource(construct)

Check whether the given construct is a Resource.

Parameters

construct (IConstruct) –

Return type

bool