JobProps
- class aws_cdk.aws_glue_alpha.JobProps(*, executable, connections=None, continuous_logging=None, default_arguments=None, description=None, enable_profiling_metrics=None, execution_class=None, job_name=None, max_capacity=None, max_concurrent_runs=None, max_retries=None, notify_delay_after=None, role=None, security_configuration=None, spark_ui=None, tags=None, timeout=None, worker_count=None, worker_type=None)
Bases:
object
(experimental) Construction properties for
Job
.- Parameters:
executable (
JobExecutable
) – (experimental) The job’s executable properties.connections (
Optional
[Sequence
[IConnection
]]) – (experimental) The ``Connection``s used for this job. Connections are used to connect to other AWS Service or resources within a VPC. Default: [] - no connections are added to the jobcontinuous_logging (
Union
[ContinuousLoggingProps
,Dict
[str
,Any
],None
]) – (experimental) Enables continuous logging with the specified props. Default: - continuous logging is disabled.default_arguments (
Optional
[Mapping
[str
,str
]]) – (experimental) The default arguments for this job, specified as name-value pairs. Default: - no argumentsdescription (
Optional
[str
]) – (experimental) The description of the job. Default: - no valueenable_profiling_metrics (
Optional
[bool
]) – (experimental) Enables the collection of metrics for job profiling. Equivalent to a job parameter--enable-metrics
. Default: - no profiling metrics emitted.execution_class (
Optional
[ExecutionClass
]) – (experimental) The ExecutionClass whether the job is run with a standard or flexible execution class. Default: - STANDARDjob_name (
Optional
[str
]) – (experimental) The name of the job. Default: - a name is automatically generatedmax_capacity (
Union
[int
,float
,None
]) – (experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead. Default: - 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shellmax_concurrent_runs (
Union
[int
,float
,None
]) – (experimental) The maximum number of concurrent runs allowed for the job. An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit. Default: 1max_retries (
Union
[int
,float
,None
]) – (experimental) The maximum number of times to retry this job after a job run fails. Default: 0notify_delay_after (
Optional
[Duration
]) – (experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification. Default: - no delay notificationsrole (
Optional
[IRole
]) – (experimental) The IAM role assumed by Glue to run this job. If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions. Default: - a role is automatically generatedsecurity_configuration (
Optional
[ISecurityConfiguration
]) – (experimental) TheSecurityConfiguration
to use for this job. Default: - no security configuration.spark_ui (
Union
[SparkUIProps
,Dict
[str
,Any
],None
]) – (experimental) Enables the Spark UI debugging and monitoring with the specified props. Default: - Spark UI debugging and monitoring is disabled.tags (
Optional
[Mapping
[str
,str
]]) – (experimental) The tags to add to the resources on which the job runs. Default: {} - no tagstimeout (
Optional
[Duration
]) – (experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. Default: cdk.Duration.hours(48)worker_count (
Union
[int
,float
,None
]) – (experimental) The number of workers of a definedWorkerType
that are allocated when a job runs. Default: - differs based on specific Glue version/worker typeworker_type (
Optional
[WorkerType
]) – (experimental) The type of predefined worker that is allocated when a job runs. Default: - differs based on specific Glue version
- Stability:
experimental
- ExampleMetadata:
infused
Example:
glue.Job(self, "EnableSparkUI", job_name="EtlJobWithSparkUIPrefix", spark_uI=glue.SparkUIProps( enabled=True ), executable=glue.JobExecutable.python_etl( glue_version=glue.GlueVersion.V3_0, python_version=glue.PythonVersion.THREE, script=glue.Code.from_asset(path.join(__dirname, "job-script", "hello_world.py")) ) )
Attributes
- connections
(experimental) The ``Connection``s used for this job.
Connections are used to connect to other AWS Service or resources within a VPC.
- Default:
[] - no connections are added to the job
- Stability:
experimental
- continuous_logging
(experimental) Enables continuous logging with the specified props.
- Default:
continuous logging is disabled.
- See:
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
- Stability:
experimental
- default_arguments
(experimental) The default arguments for this job, specified as name-value pairs.
- Default:
no arguments
- See:
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters
- Stability:
experimental
- description
(experimental) The description of the job.
- Default:
no value
- Stability:
experimental
- enable_profiling_metrics
(experimental) Enables the collection of metrics for job profiling.
Equivalent to a job parameter
--enable-metrics
.- Default:
no profiling metrics emitted.
- See:
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
- Stability:
experimental
- executable
(experimental) The job’s executable properties.
- Stability:
experimental
- execution_class
(experimental) The ExecutionClass whether the job is run with a standard or flexible execution class.
- Default:
STANDARD
- See:
- Stability:
experimental
- job_name
(experimental) The name of the job.
- Default:
a name is automatically generated
- Stability:
experimental
- max_capacity
(experimental) The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.
- Default:
10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell
- Stability:
experimental
- max_concurrent_runs
(experimental) The maximum number of concurrent runs allowed for the job.
An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
- Default:
1
- Stability:
experimental
- max_retries
(experimental) The maximum number of times to retry this job after a job run fails.
- Default:
0
- Stability:
experimental
- notify_delay_after
(experimental) The number of minutes to wait after a job run starts, before sending a job run delay notification.
- Default:
no delay notifications
- Stability:
experimental
- role
(experimental) The IAM role assumed by Glue to run this job.
If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.
- Default:
a role is automatically generated
- See:
https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html
- Stability:
experimental
- security_configuration
(experimental) The
SecurityConfiguration
to use for this job.- Default:
no security configuration.
- Stability:
experimental
- spark_ui
(experimental) Enables the Spark UI debugging and monitoring with the specified props.
- Default:
Spark UI debugging and monitoring is disabled.
- See:
https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
- Stability:
experimental
- tags
(experimental) The tags to add to the resources on which the job runs.
- Default:
{} - no tags
- Stability:
experimental
- timeout
(experimental) The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
- Default:
cdk.Duration.hours(48)
- Stability:
experimental
- worker_count
(experimental) The number of workers of a defined
WorkerType
that are allocated when a job runs.- Default:
differs based on specific Glue version/worker type
- Stability:
experimental
- worker_type
(experimental) The type of predefined worker that is allocated when a job runs.
- Default:
differs based on specific Glue version
- Stability:
experimental