Configuring all AWS calls to go through your VPC - AWS Glue

Configuring all AWS calls to go through your VPC

The special job parameter disable-proxy allows you to disable the service proxy to force all such calls through your VPC. AWS Glue uses a local proxy to send traffic through the AWS Glue VPC to download scripts and libraries and to send requests to CloudWatch for publishing logs and metrics. This proxy allows the job to function normally even if your VPC doesn't configure a proper route to Amazon S3 and CloudWatch. AWS Glue now offers a parameter for you to turn off this behavior and to force all AWS calls originating from your script to obey your network control policies. For more information, see Special Parameters Used by AWS Glue.

Note

When using this feature, you need to ensure that your VPC has configured a route to Amazon S3, AWS Glue and CloudWatch through a NAT or service VPC endpoint. Otherwise the job can fail, or the job is not able to publish continuous log and job metrics.

Example usage

Create an AWS Glue job with disable-proxy:

aws glue create-job \ --name no-proxy-job \ --role GlueDefaultRole \ --command "Name=glueetl,ScriptLocation=s3://my-bucket/glue-script.py" \ --connections Connections="traffic-monitored-connection" \ --default-arguments '{"--disable-proxy" : "true"}'