CreateProcessingJob - Amazon SageMaker


Creates a processing job.

Request Syntax

{ "AppSpecification": { "ContainerArguments": [ "string" ], "ContainerEntrypoint": [ "string" ], "ImageUri": "string" }, "Environment": { "string" : "string" }, "ExperimentConfig": { "ExperimentName": "string", "RunName": "string", "TrialComponentDisplayName": "string", "TrialName": "string" }, "NetworkConfig": { "EnableInterContainerTrafficEncryption": boolean, "EnableNetworkIsolation": boolean, "VpcConfig": { "SecurityGroupIds": [ "string" ], "Subnets": [ "string" ] } }, "ProcessingInputs": [ { "AppManaged": boolean, "DatasetDefinition": { "AthenaDatasetDefinition": { "Catalog": "string", "Database": "string", "KmsKeyId": "string", "OutputCompression": "string", "OutputFormat": "string", "OutputS3Uri": "string", "QueryString": "string", "WorkGroup": "string" }, "DataDistributionType": "string", "InputMode": "string", "LocalPath": "string", "RedshiftDatasetDefinition": { "ClusterId": "string", "ClusterRoleArn": "string", "Database": "string", "DbUser": "string", "KmsKeyId": "string", "OutputCompression": "string", "OutputFormat": "string", "OutputS3Uri": "string", "QueryString": "string" } }, "InputName": "string", "S3Input": { "LocalPath": "string", "S3CompressionType": "string", "S3DataDistributionType": "string", "S3DataType": "string", "S3InputMode": "string", "S3Uri": "string" } } ], "ProcessingJobName": "string", "ProcessingOutputConfig": { "KmsKeyId": "string", "Outputs": [ { "AppManaged": boolean, "FeatureStoreOutput": { "FeatureGroupName": "string" }, "OutputName": "string", "S3Output": { "LocalPath": "string", "S3UploadMode": "string", "S3Uri": "string" } } ] }, "ProcessingResources": { "ClusterConfig": { "InstanceCount": number, "InstanceType": "string", "VolumeKmsKeyId": "string", "VolumeSizeInGB": number } }, "RoleArn": "string", "StoppingCondition": { "MaxRuntimeInSeconds": number }, "Tags": [ { "Key": "string", "Value": "string" } ] }

Request Parameters

For information about the parameters that are common to all actions, see Common Parameters.

The request accepts the following data in JSON format.


Configures the processing job to run a specified Docker container image.

Type: AppSpecification object

Required: Yes


The environment variables to set in the Docker container. Up to 100 key and values entries in the map are supported.

Type: String to string map

Map Entries: Maximum number of 100 items.

Key Length Constraints: Maximum length of 256.

Key Pattern: [a-zA-Z_][a-zA-Z0-9_]*

Value Length Constraints: Maximum length of 256.

Value Pattern: [\S\s]*

Required: No


Associates a SageMaker job as a trial component with an experiment and trial. Specified when you call the following APIs:

Type: ExperimentConfig object

Required: No


Networking options for a processing job, such as whether to allow inbound and outbound network calls to and from processing containers, and the VPC subnets and security groups to use for VPC-enabled processing jobs.

Type: NetworkConfig object

Required: No


An array of inputs configuring the data to download into the processing container.

Type: Array of ProcessingInput objects

Array Members: Minimum number of 0 items. Maximum number of 10 items.

Required: No


The name of the processing job. The name must be unique within an AWS Region in the AWS account.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 63.

Pattern: ^[a-zA-Z0-9](-*[a-zA-Z0-9]){0,62}

Required: Yes


Output configuration for the processing job.

Type: ProcessingOutputConfig object

Required: No


Identifies the resources, ML compute instances, and ML storage volumes to deploy for a processing job. In distributed training, you specify more than one instance.

Type: ProcessingResources object

Required: Yes


The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.

Type: String

Length Constraints: Minimum length of 20. Maximum length of 2048.

Pattern: ^arn:aws[a-z\-]*:iam::\d{12}:role/?[a-zA-Z_0-9+=,.@\-_/]+$

Required: Yes


The time limit for how long the processing job is allowed to run.

Type: ProcessingStoppingCondition object

Required: No


(Optional) An array of key-value pairs. For more information, see Using Cost Allocation Tags in the AWS Billing and Cost Management User Guide.

Type: Array of Tag objects

Array Members: Minimum number of 0 items. Maximum number of 50 items.

Required: No

Response Syntax

{ "ProcessingJobArn": "string" }

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.


The Amazon Resource Name (ARN) of the processing job.

Type: String

Length Constraints: Maximum length of 256.

Pattern: arn:aws[a-z\-]*:sagemaker:[a-z0-9\-]*:[0-9]{12}:processing-job/.*


For information about the errors that are common to all actions, see Common Errors.


Resource being accessed is in use.

HTTP Status Code: 400


You have exceeded an SageMaker resource limit. For example, you might have too many training jobs created.

HTTP Status Code: 400


Resource being access is not found.

HTTP Status Code: 400

See Also

For more information about using this API in one of the language-specific AWS SDKs, see the following: