Migrating from dev endpoints to interactive sessions - AWS Glue

Migrating from dev endpoints to interactive sessions

Use the following checklist to determine the appropriate method to migrate from dev endpoints to interactive sessions.

Does your script depend on AWS Glue 0.9 or 1.0 specific features (for example, HDFS, YARN, etc.)?

If the answer is yes, see Migrating AWS Glue jobs to AWS Glue version 3.0. to learn how to migrate from Glue 0.9 or 1.0 to Glue 3.0 and later.

Which method do you use to access your dev endpoint?

If you use this method Then do this
SageMaker notebook, Jupyter notebook, or JupyterLab Migrate to AWS Glue Studio notebook by downloading .ipynb files on Jupyter and create a new AWS Glue Studio notebook job by uploading the .ipynb file. Alternatively, you can also use SageMaker Studio and select the AWS Glue kernel.
Zeppelin notebook Convert the notebook to a Jupyter notebook manually by copying and pasting code or automatically using a third-party converter such as ze2nb. Then, use the notebook in AWS Glue Studio notebook or SageMaker Studio.
IDE See Author AWS Glue jobs with PyCharm using AWS Glue interactive sessions, or Using interactive sessions with Microsoft Visual Studio Code.

Install the aws-glue-session package locally, then run the following command:

  • For Python: jupyter console --kernal glue_pyspark

  • For Scala: jupyter console --kernal glue_spark

SSH No corresponding option on interactive sessions. Alternatively, you can use a Docker image. To learn more, see Developing using a Docker image.