Package software.amazon.awscdk.services.stepfunctions.tasks

Interface SparkSubmitJobDriver

All Superinterfaces:

software.amazon.jsii.JsiiSerializable

All Known Implementing Classes:

SparkSubmitJobDriver.Jsii$Proxy

@Generated(value="jsii-pacmak/1.84.0 (build 5404dcf)",
           date="2023-06-19T16:30:48.210Z")
@Stability(Stable)
public interface SparkSubmitJobDriver
extends software.amazon.jsii.JsiiSerializable

The information about job driver for Spark submit.

Example:

 EmrContainersStartJobRun.Builder.create(this, "EMR Containers Start Job Run")
         .virtualCluster(VirtualClusterInput.fromVirtualClusterId("de92jdei2910fwedz"))
         .releaseLabel(ReleaseLabel.EMR_6_2_0)
         .jobName("EMR-Containers-Job")
         .jobDriver(JobDriver.builder()
                 .sparkSubmitJobDriver(SparkSubmitJobDriver.builder()
                         .entryPoint(TaskInput.fromText("local:///usr/lib/spark/examples/src/main/python/pi.py"))
                         .build())
                 .build())
         .applicationConfig(List.of(ApplicationConfiguration.builder()
                 .classification(Classification.SPARK_DEFAULTS)
                 .properties(Map.of(
                         "spark.executor.instances", "1",
                         "spark.executor.memory", "512M"))
                 .build()))
         .build();
 

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static final class	SparkSubmitJobDriver.Builder	A builder for SparkSubmitJobDriver
static final class	SparkSubmitJobDriver.Jsii$Proxy	An implementation for SparkSubmitJobDriver

Method Summary

Modifier and Type	Method	Description
static SparkSubmitJobDriver.Builder	builder()
TaskInput	getEntryPoint()	The entry point of job application.
default TaskInput	getEntryPointArguments()	The arguments for a job application in a task input object containing an array of strings.
default String	getSparkSubmitParameters()	The Spark submit parameters that are used for job runs.

Methods inherited from interface software.amazon.jsii.JsiiSerializable

$jsii$toJson

Method Details

getEntryPoint

@Stability(Stable)
@NotNull
TaskInput getEntryPoint()

The entry point of job application.

Length Constraints: Minimum length of 1. Maximum length of 256.

getEntryPointArguments

@Stability(Stable)
@Nullable
default TaskInput getEntryPointArguments()

The arguments for a job application in a task input object containing an array of strings.

Length Constraints: Minimum length of 1. Maximum length of 10280.

Default: - No arguments defined

getSparkSubmitParameters

@Stability(Stable)
@Nullable
default String getSparkSubmitParameters()

The Spark submit parameters that are used for job runs.

Length Constraints: Minimum length of 1. Maximum length of 102400.

Default: - No spark submit parameters

builder

@Stability(Stable)
static SparkSubmitJobDriver.Builder builder()

Returns:

a SparkSubmitJobDriver.Builder of SparkSubmitJobDriver