interface JobProps
Language | Type name |
---|---|
.NET | Amazon.CDK.AWS.Glue.JobProps |
Java | software.amazon.awscdk.services.glue.JobProps |
Python | aws_cdk.aws_glue.JobProps |
TypeScript (source) | @aws-cdk/aws-glue ยป JobProps |
Construction properties for {@link Job}.
Example
declare const bucket: s3.Bucket;
new glue.Job(this, 'PythonShellJob', {
executable: glue.JobExecutable.pythonShell({
glueVersion: glue.GlueVersion.V1_0,
pythonVersion: glue.PythonVersion.THREE,
script: glue.Code.fromBucket(bucket, 'script.py'),
}),
description: 'an example Python Shell job',
});
Properties
Name | Type | Description |
---|---|---|
executable | Job | The job's executable properties. |
connections? | IConnection [] | The {@link Connection}s used for this job. |
continuous | Continuous | Enables continuous logging with the specified props. |
default | { [string]: string } | The default arguments for this job, specified as name-value pairs. |
description? | string | The description of the job. |
enable | boolean | Enables the collection of metrics for job profiling. |
job | string | The name of the job. |
max | number | The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. |
max | number | The maximum number of concurrent runs allowed for the job. |
max | number | The maximum number of times to retry this job after a job run fails. |
notify | Duration | The number of minutes to wait after a job run starts, before sending a job run delay notification. |
role? | IRole | The IAM role assumed by Glue to run this job. |
security | ISecurity | The {@link SecurityConfiguration} to use for this job. |
spark | Spark | Enables the Spark UI debugging and monitoring with the specified props. |
tags? | { [string]: string } | The tags to add to the resources on which the job runs. |
timeout? | Duration | The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. |
worker | number | The number of workers of a defined {@link WorkerType} that are allocated when a job runs. |
worker | Worker | The type of predefined worker that is allocated when a job runs. |
executable
Type:
Job
The job's executable properties.
connections?
Type:
IConnection
[]
(optional, default: [] - no connections are added to the job)
The {@link Connection}s used for this job.
Connections are used to connect to other AWS Service or resources within a VPC.
continuousLogging?
Type:
Continuous
(optional, default: continuous logging is disabled.)
Enables continuous logging with the specified props.
See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
defaultArguments?
Type:
{ [string]: string }
(optional, default: no arguments)
The default arguments for this job, specified as name-value pairs.
See also: [https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html for a list of reserved parameters)
description?
Type:
string
(optional, default: no value)
The description of the job.
enableProfilingMetrics?
Type:
boolean
(optional, default: no profiling metrics emitted.)
Enables the collection of metrics for job profiling.
See also: [--enable-metrics
at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html](--enable-metrics
at https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html)
jobName?
Type:
string
(optional, default: a name is automatically generated)
The name of the job.
maxCapacity?
Type:
number
(optional, default: 10 when job type is Apache Spark ETL or streaming, 0.0625 when job type is Python shell)
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
Cannot be used for Glue version 2.0 and later - workerType and workerCount should be used instead.
maxConcurrentRuns?
Type:
number
(optional, default: 1)
The maximum number of concurrent runs allowed for the job.
An error is returned when this threshold is reached. The maximum value you can specify is controlled by a service limit.
maxRetries?
Type:
number
(optional, default: 0)
The maximum number of times to retry this job after a job run fails.
notifyDelayAfter?
Type:
Duration
(optional, default: no delay notifications)
The number of minutes to wait after a job run starts, before sending a job run delay notification.
role?
Type:
IRole
(optional, default: a role is automatically generated)
The IAM role assumed by Glue to run this job.
If providing a custom role, it needs to trust the Glue service principal (glue.amazonaws.com) and be granted sufficient permissions.
See also: https://docs.aws.amazon.com/glue/latest/dg/getting-started-access.html
securityConfiguration?
Type:
ISecurity
(optional, default: no security configuration.)
The {@link SecurityConfiguration} to use for this job.
sparkUI?
Type:
Spark
(optional, default: Spark UI debugging and monitoring is disabled.)
Enables the Spark UI debugging and monitoring with the specified props.
See also: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html
tags?
Type:
{ [string]: string }
(optional, default: {} - no tags)
The tags to add to the resources on which the job runs.
timeout?
Type:
Duration
(optional, default: cdk.Duration.hours(48))
The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
workerCount?
Type:
number
(optional, default: differs based on specific Glue version/worker type)
The number of workers of a defined {@link WorkerType} that are allocated when a job runs.
workerType?
Type:
Worker
(optional, default: differs based on specific Glue version)
The type of predefined worker that is allocated when a job runs.