GlueVersion Enum
AWS Glue version determines the versions of Apache Spark and Python that are available to the job.
Example
ts
import * as cdk from 'aws-cdk-lib';
import * as iam from 'aws-cdk-lib/aws-iam';
declare const stack: cdk.Stack;
declare const role: iam.IRole;
declare const script: glue.Code;
new glue.PySparkEtlJob(stack, 'PySparkETLJob', {
jobName: 'PySparkETLJobCustomName',
description: 'This is a description',
role,
script,
glueVersion: glue.GlueVersion.V5_1,
continuousLogging: { enabled: false },
workerConfiguration: {
workerType: glue.WorkerType.G_2X,
numberOfWorkers: 2,
},
maxConcurrentRuns: 100,
timeout: cdk.Duration.hours(2),
connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],
securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),
tags: {
FirstTagName: 'FirstTagValue',
SecondTagName: 'SecondTagValue',
XTagName: 'XTagValue',
},
maxRetries: 2,
});
Members
| Member | Description |
|---|---|
V0_9 | Glue version using Spark 2.2.1 and Python 2.7. |
V1_0 | Glue version using Spark 2.4.3, Python 2.7 and Python 3.6. |
V2_0 | Glue version using Spark 2.4.3 and Python 3.7. |
V3_0 | Glue version using Spark 3.1.1 and Python 3.7. |
V4_0 | Glue version using Spark 3.3.0 and Python 3.10. |
V5_0 | Glue version using Spark 3.5.4, Python 3.11, and Scala 2.12.18. |
V5_1 | Glue version using Spark 3.5.6, Python 3.11, and Scala 2.12.18. |
Referenced by
JobProps.glueVersionglue_versionglueVersionGlueVersionGlueVersion