Development endpoints - AWS Glue

Development endpoints

Note

The console experience for dev endpoints has been removed as of March 31, 2023. Creating, updating, and monitoring dev endpoints is still available via the Development endpoints API and AWS Glue CLI.

We strongly recommend migrating from dev endpoints to interactive sessions for the reasons listed below. For required actions on how to migrate from dev endpoints to interactive sessions, see Migrating from dev endpoints to interactive sessions.

Description Dev endpoints Interactive sessions
Glue version support Supports AWS Glue version 0.9 and 1.0 Supports AWS Glue version 2.0 and later
Dev endpoints are not available in the Asia Pacific (Jakarta) (ap-southeast-3), Middle East (UAE) (me-central-1), Europe (Spain) (eu-south-2), Europe (Zurich) (eu-central-2), or other new regions going forward Interactive sessions are not currently available in the Middle East (UAE) (me-central-1) region, but may be made available later
Access method to the Spark cluster Supports SSH, REPL shell, Jupyter notebook, IDE (e.g. PyCharm) supports AWS Glue Studio notebook, Jupyter notebook, various IDEs (for example, Visual Studio Code, PyCharm), and SageMaker notebook
Time to first query Requires 10-15 minutes to setup a Spark cluster Can take up to 1 minute to set up an ephemeral Spark cluster
Price model AWS charges for development endpoints based on the time that the endpoint is provisioned and the number of DPUs. Development endpoints do not time out. There is a 10-minute minimum billing duration for each provisioned development endpoint. Additionally, AWS charges for Jupyter notebook on Amazon EC2 instances, and SageMaker notebooks when you configure them with dev endpoints. AWS charges for interactive sessions based on the time that the session is active and the number of DPUs. interactive sessions have configurable idle timeouts. AWS Glue Studio notebooks provide a built-in interface for interactive sessions and are offered at no additional cost. There is a 1-minute minimum billing duration for each interactive session. AWS Glue Studio notebooks provide a built-in interface for interactive sessions and are offered at no additional cost
Console experience Only available via the CLI and API Available through the AWS Glue console, CLI, and APIs

The following sections provide information on using dev endpoints to develop jobs in AWS Glue version 1.0.