EmrContainersStartJobRunProps¶

class aws_cdk.aws_stepfunctions_tasks.EmrContainersStartJobRunProps(*, comment=None, heartbeat=None, input_path=None, integration_pattern=None, output_path=None, result_path=None, result_selector=None, timeout=None, job_driver, release_label, virtual_cluster, application_config=None, execution_role=None, job_name=None, monitoring=None, tags=None)¶

Bases: aws_cdk.aws_stepfunctions.TaskStateBaseProps

The props for a EMR Containers StartJobRun Task.

Parameters

comment (Optional[str]) – An optional description for this state. Default: - No comment
heartbeat (Optional[Duration]) – Timeout for the heartbeat. Default: - None
input_path (Optional[str]) – JSONPath expression to select part of the state to be the input to this state. May also be the special value JsonPath.DISCARD, which will cause the effective input to be the empty object {}. Default: - The entire task input (JSON path ‘$’)
integration_pattern (Optional[IntegrationPattern]) – AWS Step Functions integrates with services directly in the Amazon States Language. You can control these AWS services using service integration patterns Default: - IntegrationPattern.REQUEST_RESPONSE for most tasks. IntegrationPattern.RUN_JOB for the following exceptions: BatchSubmitJob, EmrAddStep, EmrCreateCluster, EmrTerminationCluster, and EmrContainersStartJobRun.
output_path (Optional[str]) – JSONPath expression to select select a portion of the state output to pass to the next state. May also be the special value JsonPath.DISCARD, which will cause the effective output to be the empty object {}. Default: - The entire JSON node determined by the state input, the task result, and resultPath is passed to the next state (JSON path ‘$’)
result_path (Optional[str]) – JSONPath expression to indicate where to inject the state’s output. May also be the special value JsonPath.DISCARD, which will cause the state’s input to become its output. Default: - Replaces the entire input with the result (JSON path ‘$’)
result_selector (Optional[Mapping[str, Any]]) – The JSON that will replace the state’s raw result and become the effective result before ResultPath is applied. You can use ResultSelector to create a payload with values that are static or selected from the state’s raw result. Default: - None
timeout (Optional[Duration]) – Timeout for the state machine. Default: - None
job_driver (Union[JobDriver, Dict[str, Any]]) – The job driver for the job run.
release_label (ReleaseLabel) – The Amazon EMR release version to use for the job run.
virtual_cluster (VirtualClusterInput) – The ID of the virtual cluster where the job will be run.
application_config (Optional[Sequence[Union[ApplicationConfiguration, Dict[str, Any]]]]) – The configurations for the application running in the job run. Maximum of 100 items Default: - No application config
execution_role (Optional[IRole]) – The execution role for the job run. If virtualClusterId is from a JSON input path, an execution role must be provided. If an execution role is provided, follow the documentation to update the role trust policy. Default: - Automatically generated only when the provided virtualClusterId is not an encoded JSON path
job_name (Optional[str]) – The name of the job run. Default: - No job run name
monitoring (Union[Monitoring, Dict[str, Any], None]) – Configuration for monitoring the job run. Default: - logging enabled and resources automatically generated if monitoring.logging is set to true
tags (Optional[Mapping[str, str]]) – The tags assigned to job runs. Default: - None

ExampleMetadata

infused

Example:

tasks.EmrContainersStartJobRun(self, "EMR Containers Start Job Run",
    virtual_cluster=tasks.VirtualClusterInput.from_virtual_cluster_id("de92jdei2910fwedz"),
    release_label=tasks.ReleaseLabel.EMR_6_2_0,
    job_name="EMR-Containers-Job",
    job_driver=tasks.JobDriver(
        spark_submit_job_driver=tasks.SparkSubmitJobDriver(
            entry_point=sfn.TaskInput.from_text("local:///usr/lib/spark/examples/src/main/python/pi.py")
        )
    ),
    application_config=[tasks.ApplicationConfiguration(
        classification=tasks.Classification.SPARK_DEFAULTS,
        properties={
            "spark.executor.instances": "1",
            "spark.executor.memory": "512M"
        }
    )]
)

Attributes

application_config¶

The configurations for the application running in the job run.

Maximum of 100 items

Default

No application config

See

https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_Configuration.html

Return type

Optional[List[ApplicationConfiguration]]

comment¶

An optional description for this state.

Default

No comment

Return type

Optional[str]

execution_role¶

The execution role for the job run.

If virtualClusterId is from a JSON input path, an execution role must be provided. If an execution role is provided, follow the documentation to update the role trust policy.

Default

Automatically generated only when the provided virtualClusterId is not an encoded JSON path

See

https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-trust-policy.html

Return type

Optional[IRole]

heartbeat¶

Timeout for the heartbeat.

Default

None

Return type

Optional[Duration]

input_path¶

JSONPath expression to select part of the state to be the input to this state.

May also be the special value JsonPath.DISCARD, which will cause the effective input to be the empty object {}.

Default

The entire task input (JSON path ‘$’)

Return type

Optional[str]

integration_pattern¶

AWS Step Functions integrates with services directly in the Amazon States Language.

You can control these AWS services using service integration patterns

Default

IntegrationPattern.REQUEST_RESPONSE for most tasks.

IntegrationPattern.RUN_JOB for the following exceptions: BatchSubmitJob, EmrAddStep, EmrCreateCluster, EmrTerminationCluster, and EmrContainersStartJobRun.

See: https://docs.aws.amazon.com/step-functions/latest/dg/connect-to-resource.html#connect-wait-token
Return type: Optional[IntegrationPattern]

job_driver¶

The job driver for the job run.

See: https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_JobDriver.html
Return type: JobDriver

job_name¶

The name of the job run.

Default

No job run name

Return type

Optional[str]

monitoring¶

Configuration for monitoring the job run.

Default

logging enabled and resources automatically generated if monitoring.logging is set to true

See

https://docs.aws.amazon.com/emr-on-eks/latest/APIReference/API_MonitoringConfiguration.html

Return type

Optional[Monitoring]

output_path¶

JSONPath expression to select select a portion of the state output to pass to the next state.

May also be the special value JsonPath.DISCARD, which will cause the effective output to be the empty object {}.

Default

The entire JSON node determined by the state input, the task result,

and resultPath is passed to the next state (JSON path ‘$’)

Return type: Optional[str]

release_label¶

The Amazon EMR release version to use for the job run.

Return type: ReleaseLabel

result_path¶

JSONPath expression to indicate where to inject the state’s output.

May also be the special value JsonPath.DISCARD, which will cause the state’s input to become its output.

Default

Replaces the entire input with the result (JSON path ‘$’)

Return type

Optional[str]

result_selector¶

The JSON that will replace the state’s raw result and become the effective result before ResultPath is applied.

You can use ResultSelector to create a payload with values that are static or selected from the state’s raw result.

Default

None

See

https://docs.aws.amazon.com/step-functions/latest/dg/input-output-inputpath-params.html#input-output-resultselector

Return type

Optional[Mapping[str, Any]]

tags¶

The tags assigned to job runs.

Default

None

Return type

Optional[Mapping[str, str]]

timeout¶

Timeout for the state machine.

Default

None

Return type

Optional[Duration]

virtual_cluster¶

The ID of the virtual cluster where the job will be run.

Return type: VirtualClusterInput