Interface ICfnJobProps
Properties for defining a CfnJob
.
Namespace: Amazon.CDK.AWS.Glue
Assembly: Amazon.CDK.AWS.Glue.dll
Syntax (csharp)
public interface ICfnJobProps
Syntax (vb)
Public Interface ICfnJobProps
Remarks
Link: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-job.html
ExampleMetadata: fixture=_generated
Examples
// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
using Amazon.CDK.AWS.Glue;
var defaultArguments;
var nonOverridableArguments;
var tags;
var cfnJobProps = new CfnJobProps {
Command = new JobCommandProperty {
Name = "name",
PythonVersion = "pythonVersion",
Runtime = "runtime",
ScriptLocation = "scriptLocation"
},
Role = "role",
// the properties below are optional
AllocatedCapacity = 123,
Connections = new ConnectionsListProperty {
Connections = new [] { "connections" }
},
DefaultArguments = defaultArguments,
Description = "description",
ExecutionClass = "executionClass",
ExecutionProperty = new ExecutionPropertyProperty {
MaxConcurrentRuns = 123
},
GlueVersion = "glueVersion",
LogUri = "logUri",
MaxCapacity = 123,
MaxRetries = 123,
Name = "name",
NonOverridableArguments = nonOverridableArguments,
NotificationProperty = new NotificationPropertyProperty {
NotifyDelayAfter = 123
},
NumberOfWorkers = 123,
SecurityConfiguration = "securityConfiguration",
Tags = tags,
Timeout = 123,
WorkerType = "workerType"
};
Synopsis
Properties
AllocatedCapacity | This parameter is no longer supported. Use |
Command | The code that executes a job. |
Connections | The connections used for this job. |
DefaultArguments | The default arguments for this job, specified as name-value pairs. |
Description | A description of the job. |
ExecutionClass | Indicates whether the job is run with a standard or flexible execution class. |
ExecutionProperty | The maximum number of concurrent runs that are allowed for this job. |
GlueVersion | Glue version determines the versions of Apache Spark and Python that AWS Glue supports. |
LogUri | This field is reserved for future use. |
MaxCapacity | The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs. |
MaxRetries | The maximum number of times to retry this job after a JobRun fails. |
Name | The name you assign to this job definition. |
NonOverridableArguments | Non-overridable arguments for this job, specified as name-value pairs. |
NotificationProperty | Specifies configuration properties of a notification. |
NumberOfWorkers | The number of workers of a defined |
Role | The name or Amazon Resource Name (ARN) of the IAM role associated with this job. |
SecurityConfiguration | The name of the |
Tags | The tags to use with this job. |
Timeout | The job timeout in minutes. |
WorkerType | The type of predefined worker that is allocated when a job runs. |
Properties
AllocatedCapacity
This parameter is no longer supported. Use MaxCapacity
instead.
virtual Nullable<double> AllocatedCapacity { get; }
Property Value
System.Nullable<System.Double>
Remarks
The number of capacity units that are allocated to this job.
Command
The code that executes a job.
object Command { get; }
Property Value
System.Object
Remarks
Connections
The connections used for this job.
virtual object Connections { get; }
Property Value
System.Object
Remarks
DefaultArguments
The default arguments for this job, specified as name-value pairs.
virtual object DefaultArguments { get; }
Property Value
System.Object
Remarks
You can specify arguments here that your own job-execution script consumes, in addition to arguments that AWS Glue itself consumes.
For information about how to specify and consume your own job arguments, see Calling AWS Glue APIs in Python in the AWS Glue Developer Guide .
For information about the key-value pairs that AWS Glue consumes to set up your job, see Special Parameters Used by AWS Glue in the AWS Glue Developer Guide .
Description
A description of the job.
virtual string Description { get; }
Property Value
System.String
Remarks
ExecutionClass
Indicates whether the job is run with a standard or flexible execution class.
virtual string ExecutionClass { get; }
Property Value
System.String
Remarks
The standard execution class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.
The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.
Only jobs with AWS Glue version 3.0 and above and command type glueetl
will be allowed to set ExecutionClass
to FLEX
. The flexible execution class is available for Spark jobs.
ExecutionProperty
The maximum number of concurrent runs that are allowed for this job.
virtual object ExecutionProperty { get; }
Property Value
System.Object
Remarks
GlueVersion
Glue version determines the versions of Apache Spark and Python that AWS Glue supports.
virtual string GlueVersion { get; }
Property Value
System.String
Remarks
The Python version indicates the version supported for jobs of type Spark.
For more information about the available AWS Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.
Jobs that are created without specifying a Glue version default to Glue 0.9.
LogUri
This field is reserved for future use.
virtual string LogUri { get; }
Property Value
System.String
Remarks
MaxCapacity
The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
virtual Nullable<double> MaxCapacity { get; }
Property Value
System.Nullable<System.Double>
Remarks
A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.
Do not set Max Capacity
if using WorkerType
and NumberOfWorkers
.
The value that can be allocated for MaxCapacity
depends on whether you are running a Python shell job or an Apache Spark ETL job:
MaxRetries
The maximum number of times to retry this job after a JobRun fails.
virtual Nullable<double> MaxRetries { get; }
Property Value
System.Nullable<System.Double>
Remarks
Name
The name you assign to this job definition.
virtual string Name { get; }
Property Value
System.String
Remarks
NonOverridableArguments
Non-overridable arguments for this job, specified as name-value pairs.
virtual object NonOverridableArguments { get; }
Property Value
System.Object
Remarks
NotificationProperty
Specifies configuration properties of a notification.
virtual object NotificationProperty { get; }
Property Value
System.Object
Remarks
NumberOfWorkers
The number of workers of a defined workerType
that are allocated when a job runs.
virtual Nullable<double> NumberOfWorkers { get; }
Property Value
System.Nullable<System.Double>
Remarks
The maximum number of workers you can define are 299 for G.1X
, and 149 for G.2X
.
Role
The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
string Role { get; }
Property Value
System.String
Remarks
SecurityConfiguration
The name of the SecurityConfiguration
structure to be used with this job.
virtual string SecurityConfiguration { get; }
Property Value
System.String
Remarks
Tags
The tags to use with this job.
virtual object Tags { get; }
Property Value
System.Object
Remarks
Timeout
The job timeout in minutes.
virtual Nullable<double> Timeout { get; }
Property Value
System.Nullable<System.Double>
Remarks
This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status. The default is 2,880 minutes (48 hours).
WorkerType
The type of predefined worker that is allocated when a job runs.
virtual string WorkerType { get; }
Property Value
System.String
Remarks
Accepts a value of Standard, G.1X, or G.2X.