AWS Glue version support policy - AWS Glue

AWS Glue version support policy

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. An AWS Glue job contains the business logic that performs the data integration work in AWS Glue. There are two types of jobs in AWS Glue: Spark (batch and streaming) and Python shell. A Spark job is run in an Apache Spark environment managed by AWS Glue. A Python shell job runs Python scripts as a shell. When you define your job, you specify the AWS Glue version which configures the Spark and Python version in the runtime environment. For example: an AWS Glue version 2.0 job supports Spark 2.4.3 and Python 3.7.

Support policy

Occasionally AWS Glue discontinues support for old AWS Glue versions. You can continue to create, update and run jobs on those versions. However, jobs running on deprecated versions are no longer eligible for technical support. AWS Glue will no longer apply security patches or other updates to deprecated versions. AWS Glue will also not honor SLAs when jobs are run on deprecated versions.

The following AWS Glue versions have reached or are scheduled for end of support. End of support starts at midnight (Pacific time zone) on the specified date.

Type Glue version End of support
Spark Spark 2.2, Scala 2 (Glue version 0.9) 6/1/2022
Spark Spark 2.2, Python 2 (Glue version 0.9) 6/1/2022
Spark Spark 2.4, Python 2 (Glue version 1.0) 6/1/2022
Spark Spark 2.4, Python 3 (Glue version 1.0) 9/30/2022
Spark Spark 2.4, Scala 2 (Glue version 1.0) 9/30/2022
Spark Glue version 2.0 1/31/2024
Type Python version End of support
Python shell Python 2 (Glue Version 1.0) 6/1/2022
Type Notebook version End of support
Development endpoint Zeppelin notebook 9/30/2022

AWS strongly recommends that you migrate your jobs to supported versions.

For information on migrating your Spark jobs to the latest AWS Glue version, see Migrating AWS Glue jobs to AWS Glue version 4.0.

For migrating your Python shell jobs to the latest AWS Glue version:

  • In the console, choose Python 3 (Glue Version 1.0).

  • In the CreateJob/UpdateJob API, set the GlueVersion parameter to 1.0, and the PythonVersion to 3 under the Command parameter.

  • You need to make your job script compatible with Python 3.