AWS Glue version support policy
AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development. An AWS Glue job contains the business logic that performs the data integration work in AWS Glue. There are three types of jobs in AWS Glue: Spark (batch and streaming), Ray and Python shell. When you define your job, you specify the AWS Glue version, which configures versions in the underlying Spark, Ray or Python runtime environment. For example: an AWS Glue version 2.0 Spark job supports Spark 2.4.3 and Python 3.7.
Support policy
Occasionally AWS Glue discontinues support for old AWS Glue versions. However, jobs running on deprecated versions are no longer eligible for technical support. AWS Glue will no longer apply security patches or other updates to deprecated versions. AWS Glue will also not honor SLAs when jobs are run on deprecated versions.
When end of support occurs for AWS Glue version 2.0 or later, you will not be able to create jobs, but only edit or run jobs.
The following AWS Glue versions have reached or are scheduled for end of support. End of support starts at midnight (Pacific time zone) on the specified date.
| Type | Glue version | End of support |
|---|---|---|
| Spark | Spark 2.2, Scala 2 (Glue version 0.9) | 6/1/2022 |
| Spark | Spark 2.2, Python 2 (Glue version 0.9) | 6/1/2022 |
| Spark | Spark 2.4, Python 2 (Glue version 1.0) | 6/1/2022 |
| Spark | Spark 2.4, Python 3 (Glue version 1.0) | 9/30/2022 |
| Spark | Spark 2.4, Scala 2 (Glue version 1.0) | 9/30/2022 |
| Spark | Glue version 2.0 | 1/31/2024 |
| Type | Python version | End of support |
| Python shell | Python 2 (Glue Version 1.0) | 6/1/2022 |
| Type | Notebook version | End of support |
| Development endpoint | Zeppelin notebook | 9/30/2022 |
AWS strongly recommends that you migrate your jobs to supported versions.
For information on migrating your Spark jobs to the latest AWS Glue version, see Migrating AWS Glue jobs to AWS Glue version 4.0.
For migrating your Python shell jobs to the latest AWS Glue version:
In the console, choose
Python 3 (Glue Version 4.0).-
In the CreateJob/UpdateJob API, set the
GlueVersionparameter to2.0, and thePythonVersionto3under theCommandparameter. TheGlueVersionconfiguration does not affect the behavior of Python shell jobs, so there is no advantage to incrementingGlueVersion. You need to make your job script compatible with Python 3.
Note
All AWS Regions that were launched prior to the launch of the Jakarta, Indonesia(ap-southeast-3) Region in August 2022 have an allow list of customers who are allowed to run AWS Glue version 0.9/1.0 job runs. In these older Regions, you can create a job with a null value and it will default to version 0.9/1.0 depending on Region. For any later launched AWS Regions, you must explicitly set the AWS Glue version in the API. AWS Glue no longer accepts a null parameter. If you pass 0.9 or 1.0 in the parameter you encounter the error "Glue Version 0.9 (or) 1.0 is not supported."
