Class PySparkFlexEtlJob

(experimental) Flex Jobs class.

Inheritance
object
Resource
JobBase
Job
SparkJob
PySparkFlexEtlJob
Implements
IConstruct
IDependable
Inherited Members
Namespace: Amazon.CDK.AWS.Glue.Alpha
Assembly: Amazon.CDK.AWS.Glue.Alpha.dll
Syntax (csharp)
public class PySparkFlexEtlJob : SparkJob, IJob, IResource, IConstruct, IDependable, IEnvironmentAware, IGrantable
Syntax (vb)
Public Class PySparkFlexEtlJob Inherits SparkJob Implements IJob, IResource, IConstruct, IDependable, IEnvironmentAware, IGrantable
Remarks

Flex jobs supports Python and Scala language. The flexible execution class is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads. Flexible job runs are supported for jobs using AWS Glue version 3.0 or later and G.1X or G.2X worker types but will default to the latest version of Glue (currently Glue 3.0.)

Similar to ETL, we’ll enable these features: —enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log

Stability: Experimental

ExampleMetadata: infused

Examples
using Amazon.CDK;
             using Amazon.CDK.AWS.IAM;
             Stack stack;
             IRole role;
             Code script;

             new PySparkFlexEtlJob(stack, "ImportedJob", new PySparkFlexEtlJobProps { Role = role, Script = script });

Synopsis

Constructors

PySparkFlexEtlJob(Construct, string, IPySparkFlexEtlJobProps)

(experimental) PySparkFlexEtlJob constructor.

Properties

JobArn

(experimental) The ARN of the job.
JobName

(experimental) The name of the job.
PROPERTY_INJECTION_ID

(experimental) Uniquely identifies this class.

Constructors

PySparkFlexEtlJob(Construct, string, IPySparkFlexEtlJobProps)

(experimental) PySparkFlexEtlJob constructor.

public PySparkFlexEtlJob(Construct scope, string id, IPySparkFlexEtlJobProps props)
Parameters
scope Construct
id string
props IPySparkFlexEtlJobProps
Remarks

Stability: Experimental

Properties

JobArn

(experimental) The ARN of the job.

public override string JobArn { get; }
Property Value

string

Overrides
Remarks

Stability: Experimental

JobName

(experimental) The name of the job.

public override string JobName { get; }
Property Value

string

Overrides
Remarks

Stability: Experimental

PROPERTY_INJECTION_ID

(experimental) Uniquely identifies this class.

public static string PROPERTY_INJECTION_ID { get; }
Property Value

string

Remarks

Stability: Experimental

Implements

Constructs.IConstruct
Constructs.IDependable