Class CfnJobProps
Properties for defining a CfnJob.
Inheritance
Implements
Namespace: Amazon.CDK.AWS.Glue
Assembly: Amazon.CDK.Lib.dll
Syntax (csharp)
public class CfnJobProps : Object, ICfnJobPropsSyntax (vb)
Public Class CfnJobProps
Inherits Object
Implements ICfnJobPropsRemarks
See: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-job.html
ExampleMetadata: fixture=_generated
Examples
// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
using Amazon.CDK.AWS.Glue;
var defaultArguments;
var nonOverridableArguments;
var tags;
var cfnJobProps = new CfnJobProps {
Command = new JobCommandProperty {
Name = "name",
PythonVersion = "pythonVersion",
Runtime = "runtime",
ScriptLocation = "scriptLocation"
},
Role = "role",
// the properties below are optional
AllocatedCapacity = 123,
Connections = new ConnectionsListProperty {
Connections = new [] { "connections" }
},
DefaultArguments = defaultArguments,
Description = "description",
ExecutionClass = "executionClass",
ExecutionProperty = new ExecutionPropertyProperty {
MaxConcurrentRuns = 123
},
GlueVersion = "glueVersion",
JobMode = "jobMode",
JobRunQueuingEnabled = false,
LogUri = "logUri",
MaintenanceWindow = "maintenanceWindow",
MaxCapacity = 123,
MaxRetries = 123,
Name = "name",
NonOverridableArguments = nonOverridableArguments,
NotificationProperty = new NotificationPropertyProperty {
NotifyDelayAfter = 123
},
NumberOfWorkers = 123,
SecurityConfiguration = "securityConfiguration",
Tags = tags,
Timeout = 123,
WorkerType = "workerType"
};Synopsis
Constructors
| CfnJobProps() |
Properties
| AllocatedCapacity | This parameter is no longer supported. Use |
| Command | The code that executes a job. |
| Connections | The connections used for this job. |
| DefaultArguments | The default arguments for this job, specified as name-value pairs. |
| Description | A description of the job. |
| ExecutionClass | Indicates whether the job is run with a standard or flexible execution class. |
| ExecutionProperty | The maximum number of concurrent runs that are allowed for this job. |
| GlueVersion | Glue version determines the versions of Apache Spark and Python that AWS Glue supports. |
| JobMode | A mode that describes how a job was created. Valid values are:. |
| JobRunQueuingEnabled | Specifies whether job run queuing is enabled for the job runs for this job. |
| LogUri | This field is reserved for future use. |
| MaintenanceWindow | This field specifies a day of the week and hour for a maintenance window for streaming jobs. |
| MaxCapacity | The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. |
| MaxRetries | The maximum number of times to retry this job after a JobRun fails. |
| Name | The name you assign to this job definition. |
| NonOverridableArguments | Non-overridable arguments for this job, specified as name-value pairs. |
| NotificationProperty | Specifies configuration properties of a notification. |
| NumberOfWorkers | The number of workers of a defined |
| Role | The name or Amazon Resource Name (ARN) of the IAM role associated with this job. |
| SecurityConfiguration | The name of the |
| Tags | The tags to use with this job. |
| Timeout | The job timeout in minutes. |
| WorkerType | The type of predefined worker that is allocated when a job runs. |
Constructors
CfnJobProps()
public CfnJobProps()Properties
AllocatedCapacity
This parameter is no longer supported. Use MaxCapacity instead.
public Nullable<double> AllocatedCapacity { get; set; }Property Value
System.Nullable<System.Double>
Remarks
The number of capacity units that are allocated to this job.
Command
The code that executes a job.
public object Command { get; set; }Property Value
System.Object
Remarks
Connections
The connections used for this job.
public object Connections { get; set; }Property Value
System.Object
Remarks
DefaultArguments
The default arguments for this job, specified as name-value pairs.
public object DefaultArguments { get; set; }Property Value
System.Object
Remarks
You can specify arguments here that your own job-execution script consumes, in addition to arguments that AWS Glue itself consumes.
For information about how to specify and consume your own job arguments, see Calling AWS Glue APIs in Python in the AWS Glue Developer Guide .
For information about the key-value pairs that AWS Glue consumes to set up your job, see Special Parameters Used by AWS Glue in the AWS Glue Developer Guide .
Description
A description of the job.
public string Description { get; set; }Property Value
System.String
Remarks
ExecutionClass
Indicates whether the job is run with a standard or flexible execution class.
public string ExecutionClass { get; set; }Property Value
System.String
Remarks
The standard execution class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.
The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.
Only jobs with AWS Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX . The flexible execution class is available for Spark jobs.
ExecutionProperty
The maximum number of concurrent runs that are allowed for this job.
public object ExecutionProperty { get; set; }Property Value
System.Object
Remarks
GlueVersion
Glue version determines the versions of Apache Spark and Python that AWS Glue supports.
public string GlueVersion { get; set; }Property Value
System.String
Remarks
The Python version indicates the version supported for jobs of type Spark.
For more information about the available AWS Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.
Jobs that are created without specifying a Glue version default to the latest Glue version available.
JobMode
A mode that describes how a job was created. Valid values are:.
public string JobMode { get; set; }Property Value
System.String
Remarks
When the JobMode field is missing or null, SCRIPT is assigned as the default value.
JobRunQueuingEnabled
Specifies whether job run queuing is enabled for the job runs for this job.
public object JobRunQueuingEnabled { get; set; }Property Value
System.Object
Remarks
A value of true means job run queuing is enabled for the job runs. If false or not populated, the job runs will not be considered for queueing.
If this field does not match the value set in the job run, then the value from the job run field will be used.
LogUri
This field is reserved for future use.
public string LogUri { get; set; }Property Value
System.String
Remarks
MaintenanceWindow
This field specifies a day of the week and hour for a maintenance window for streaming jobs.
public string MaintenanceWindow { get; set; }Property Value
System.String
Remarks
AWS Glue periodically performs maintenance activities. During these maintenance windows, AWS Glue will need to restart your streaming jobs.
AWS Glue will restart the job within 3 hours of the specified maintenance window. For instance, if you set up the maintenance window for Monday at 10:00AM GMT, your jobs will be restarted between 10:00AM GMT to 1:00PM GMT.
MaxCapacity
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
public Nullable<double> MaxCapacity { get; set; }Property Value
System.Nullable<System.Double>
Remarks
A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.
Do not set Max Capacity if using WorkerType and NumberOfWorkers .
The value that can be allocated for MaxCapacity depends on whether you are running a Python shell job or an Apache Spark ETL job:
MaxRetries
The maximum number of times to retry this job after a JobRun fails.
public Nullable<double> MaxRetries { get; set; }Property Value
System.Nullable<System.Double>
Remarks
Name
The name you assign to this job definition.
public string Name { get; set; }Property Value
System.String
Remarks
NonOverridableArguments
Non-overridable arguments for this job, specified as name-value pairs.
public object NonOverridableArguments { get; set; }Property Value
System.Object
Remarks
NotificationProperty
Specifies configuration properties of a notification.
public object NotificationProperty { get; set; }Property Value
System.Object
Remarks
NumberOfWorkers
The number of workers of a defined workerType that are allocated when a job runs.
public Nullable<double> NumberOfWorkers { get; set; }Property Value
System.Nullable<System.Double>
Remarks
The maximum number of workers you can define are 299 for G.1X , and 149 for G.2X .
Role
The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
public string Role { get; set; }Property Value
System.String
Remarks
SecurityConfiguration
The name of the SecurityConfiguration structure to be used with this job.
public string SecurityConfiguration { get; set; }Property Value
System.String
Remarks
Tags
The tags to use with this job.
public object Tags { get; set; }Property Value
System.Object
Remarks
Timeout
The job timeout in minutes.
public Nullable<double> Timeout { get; set; }Property Value
System.Nullable<System.Double>
Remarks
This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).
WorkerType
The type of predefined worker that is allocated when a job runs.
public string WorkerType { get; set; }Property Value
System.String
Remarks
Accepts a value of G.1X, G.2X, G.4X, G.8X or G.025X for Spark jobs. Accepts the value Z.2X for Ray jobs.