Class SparkJob

(deprecated) Base class for different types of Spark Jobs.

Inheritance
object
Resource
JobBase
Job
SparkJob
PySparkEtlJob
PySparkFlexEtlJob
PySparkStreamingJob
ScalaSparkEtlJob
ScalaSparkFlexEtlJob
ScalaSparkStreamingJob
Implements
IConstruct
IDependable
Inherited Members
Namespace: Amazon.CDK.AWS.Glue.Alpha
Assembly: Amazon.CDK.AWS.Glue.Alpha.dll
Syntax (csharp)
[Obsolete]
public abstract class SparkJob : Job, IJob, IResource, IGrantable, IJobRef, IConstruct, IDependable, IEnvironmentAware
Syntax (vb)
<Obsolete>
Public MustInherit Class SparkJob Inherits Job Implements IJob, IResource, IGrantable, IJobRef, IConstruct, IDependable, IEnvironmentAware
Remarks

Stability: Deprecated

ExampleMetadata: fixture=_generated

Examples
// The code below shows an example of how to instantiate this type.
             // The values are placeholders you should change.
             using Amazon.CDK.AWS.Glue.Alpha;
             using Amazon.CDK.AWS.IAM;

             Role role;

             var sparkJob = SparkJob.FromJobAttributes(this, "MySparkJob", new JobAttributes {
                 JobName = "jobName",

                 // the properties below are optional
                 Role = role
             });

Synopsis

Constructors

SparkJob(Construct, string, ISparkJobProps)

(deprecated) Base class for different types of Spark Jobs.

Properties

GrantPrincipal

(deprecated) The principal to grant permissions to.
Role

(deprecated) The IAM role Glue assumes to run this job.
SparkUILoggingLocation

(deprecated) The Spark UI logs location if Spark UI monitoring and debugging is enabled.

Methods

NonExecutableCommonArguments(ISparkJobProps)

(deprecated) Register the arguments this construct manages for a Spark job.
SetupExtraCodeArguments(ISparkExtraCodeProps)

(deprecated) Register the arguments for extra {@link Code}-related properties.

Constructors

SparkJob(Construct, string, ISparkJobProps)

(deprecated) Base class for different types of Spark Jobs.

[Obsolete]
protected SparkJob(Construct scope, string id, ISparkJobProps props)
Parameters
scope Construct
id string
props ISparkJobProps
Remarks

Stability: Deprecated

Properties

GrantPrincipal

(deprecated) The principal to grant permissions to.

[Obsolete]
public override IPrincipal GrantPrincipal { get; }
Property Value

IPrincipal

Overrides
Remarks

Stability: Deprecated

Role

(deprecated) The IAM role Glue assumes to run this job.

[Obsolete]
public override IRole Role { get; }
Property Value

IRole

Overrides
Remarks

Stability: Deprecated

SparkUILoggingLocation

(deprecated) The Spark UI logs location if Spark UI monitoring and debugging is enabled.

[Obsolete]
public virtual ISparkUILoggingLocation? SparkUILoggingLocation { get; }
Property Value

ISparkUILoggingLocation

Remarks

Stability: Deprecated

See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

Methods

NonExecutableCommonArguments(ISparkJobProps)

(deprecated) Register the arguments this construct manages for a Spark job.

[Obsolete]
protected virtual void NonExecutableCommonArguments(ISparkJobProps props)
Parameters
props ISparkJobProps
Remarks

These are owned by the construct (derived from typed props). Each key is declared via {@link setManagedArgument} whether or not the current configuration emits a value, so a disabled feature (e.g. enableMetrics: false) cannot be silently re-enabled through defaultArguments.

Stability: Deprecated

SetupExtraCodeArguments(ISparkExtraCodeProps)

(deprecated) Register the arguments for extra {@link Code}-related properties.

[Obsolete]
protected virtual void SetupExtraCodeArguments(ISparkExtraCodeProps props)
Parameters
props ISparkExtraCodeProps
Remarks

Stability: Deprecated

Implements

Constructs.IConstruct
Constructs.IDependable